Play Is How Systems May Learn What Can’t Be Found

A child stacking blocks is generating facts through interaction with objects whose behavior was never cataloged. Each stack that stands and each stack that falls produces a new observation about how these particular blocks behave when placed on each other. The behavior was latent in the blocks. The observations were not available to the learner until interaction made them.

Most facts about the specific systems a deployed AI encounters are like this. A staging instance responds differently to edge-case inputs than the documentation suggests. A test harness exposes behavior the library’s reference pages never described. A development sandbox contains interactions between components that nobody wrote up because nobody had reason to. These facts do not yet exist anywhere outside the behavior of the systems themselves. Finding them requires interaction.

Reading can derive some unrecorded facts from recorded ones. A system with the right premises in its context can reason about consequences nobody wrote down. What it cannot do is discover the contingent behavior of a specific system whose behavior was never observed or recorded. No quantity of training data recovers a fact about how this particular service handles this particular input when nobody ever ran that combination.

Play, as the term is used here, is interaction that is not directed at a specific goal: doing things to see what happens, and keeping track of what did happen.

Play is the mechanism by which interaction produces facts that reading cannot find.

Systems without play are restricted to what has already been observed and written down by someone else. Systems with play can discover what nobody bothered to record.

A corpus is a record

A text corpus is a set of recorded claims. Large corpora contain many claims from many sources and many periods, and it is tempting to treat large as close to complete. A corpus can be large along several axes at once and still be finite along all of them. Every claim in it is there because some person or system, at some time, produced a record of it.

A corpus is a window onto what someone bothered to record.

Retrieval, pattern-matching, summarization, and recombination over recorded claims can produce outputs that look novel, and reasoning can extend them, but the facts they start from are the recorded ones. Training on more corpora adds more recorded claims. It does not add the ones nobody recorded.

Most facts about a system were never written down

Most of what is true about a deployed system was never written down: every code path that meets an edge case nobody described, every configuration nobody blogged about, every interaction between components nobody traced in a technical note. Somebody could find these facts if they looked. Nobody has, so they are not in the corpus. The category is large, and its shape is predictable, as three ordinary cases show.

The library. Consider a library with a published reference. The reference describes the functions, their signatures, and what they return under normal use. It does not describe what happens when a function is called with an empty list, an integer outside its stated range, a null reference, or inputs the author did not consider. Those behaviors exist. They are facts about how the library actually behaves. The reference does not document them because the reference was written before anyone tried.

The sandbox. Consider a development sandbox containing a handful of services talking to each other. The services are individually documented. The interactions between them are not. When service A fails slowly rather than fast, service B times out in a way that cascades differently than the retry logic suggests. The cascade is real. It is visible to anyone who runs the services in combination and observes the result. Nobody wrote it up, because nobody had occasion to write it up. The fact exists only in the behavior.

The repository. Consider a programmer joining a new project. The repository has a README. The README names the build command, the test command, and the entry point. The repository also has implicit conventions: which modules are the real ones and which are deprecated but not deleted, which tests are flaky, which config values are load-bearing despite appearing optional. None of this is in the README. None of it is in any file. It lives in the heads of the people who have worked on the repository long enough to know it. A new programmer does not know it. Neither does a system ingesting the repository.

Each of these cases contains facts that no amount of searching the literature can surface, because no one made them into text. They are accessible only through interaction. Running the function with the unexpected input produces the fact about how it handles the unexpected input. Running the services together produces the fact about how their failures cascade. Working in the repository for a while produces the conventions.

The facts are also mundane. Anyone who ran the code would see them. They simply have not been written, and a system whose access to the world comes entirely through text has no way to get them.

What play produces

A system with access to a sandbox and permission to interact with it can do something corpus search cannot. It can call a function and observe what happens. It can pass unusual inputs and record the responses. It can combine components and watch the combination behave. Each action produces an observation. Each observation is a new fact, generated by the system and not retrieved from anywhere else.

When that interaction is not directed at a specific goal, it is play. A child stacking blocks is not trying to verify a prediction about gravity. They are placing blocks, observing whether the stack stands, and accumulating a sense of how these blocks behave. The accumulation is the point.

What play produces differs from what reading produces in two ways, and both matter for the kind of knowledge play enables.

• It is first-hand. The system has its own record of having seen the behavior, rather than someone else’s summary. When a function returned an error on empty input during interaction, the fact is the system’s own. There is no question of whether the documentation is out of date or the blog post is wrong. The observation replaces the need for secondary sources on that particular question.
• It is specific. The observations are about the system actually in front of the agent, rather than a reference version of something similar. The library used in this project may be a version the documentation does not cover. The services in this sandbox may be configured in ways that only the person who set them up understands. The facts produced by interaction are about these systems.

Play produces observations. It does not automatically produce usable knowledge. An agent architecture still has to decide which observations to retain, which are reliable, which are local to a version or configuration, and how they should be retrieved when later work depends on them. Interaction without retention produces the same situation as not having interacted at all.

Play and directed work

Task-directed interaction produces observations within the boundary of a goal. Play produces observations outside the boundary of any immediate goal. A system calling a tool in response to a specific question is directed by that question: the call is made because the answer is needed, and whatever is observed during the call that is not relevant to the answer is discarded. A system probing a function to figure out why it fails is doing something richer than pure retrieval. It is still directed: the goal is finding the failure. Observations that do not bear on the failure go unretained.

Play has no such shaping. There is no question driving the action. A system that places a block on another block to see whether the stack stands is generating an observation that may or may not matter later. The observations accumulate. Some become relevant when a question arrives. Most do not, and they still contribute to the sense of how the system behaves that lets later questions get answered quickly.

The distinction matters because a system that only interacts when asked to interact cannot discover the facts it was not asked about. Its interaction surface is exactly the set of tool calls prompted by incoming requests. Whatever lies outside that surface stays unobserved. The facts about what happens when the function is called with an input nobody thought to ask about remain unknown until somebody asks. By then, the system is answering under time pressure with no prior observation to draw on.

Related research

Software sandboxes and game worlds share the relevant property: they are environments whose behavior can be sampled. Togelius identifies open-ended learning in virtual environments as one of the two main technical approaches to more general AI, alongside foundation models (Togelius 2024). POET demonstrates that a system can generate its own problems and solve them, producing a self-expanding curriculum without a human designing it (Wang et al. 2019). The claim here extends that line of work: interaction with an environment produces knowledge that no corpus describing the environment could have provided. The corpus, no matter how large, is finite along every axis. The interaction keeps producing.

Intrinsic-motivation approaches in reinforcement learning approximate aspects of play. An agent rewarded for prediction error seeks out states it does not yet predict well (Pathak et al. 2017). An agent rewarded for novelty seeks out states it has not yet visited (Bellemare et al. 2016). These approaches produce interaction that is not fully goal-directed in the conventional sense, and they turn something like curiosity into a specific reward function. The reward function also narrows them: a state that produces no prediction error and no novelty is invisible to the agent. Play, in the sense used here, also includes observations that would not register as interesting by any such metric but later turn out to matter.

Deployed agents skip the first week

Most deployed language-model agents have no mechanism for any of this. They have tool access, and they use the tools when prompted. Some keep notes or memory between sessions, but what they record is what a request produced. They have no separate mode in which they interact with systems without being asked to. They do not generate their own problems, they do not pursue prediction error, and they do not accumulate familiarity in the background. The facts that only undirected interaction produces remain outside their reach.

Deployment means putting a system to work in an environment it has not seen before. The staging instance, the test harness, the development sandbox, the internal tooling: none of these were in the training corpus, and none of them behave exactly like the reference versions the system consulted. The work starts now, and the unfamiliar environment is the work.

A human colleague in the same position spends the first week or so becoming familiar with the environment before taking on real work. The period might be called onboarding or ramp-up, but the activity within it is play. The new hire pulls the repository, runs the tests, reads the logs from a local run, passes unexpected inputs to functions in an isolated notebook, stands up the dev stack and watches it fail, reads the failure, and tries something different. The purpose is not to complete a task. The purpose is to build the sense of how this environment behaves that makes later task completion possible. At the end of the first week, the new hire has facts about this environment that nobody told them and that are not written down. Those facts are what let them be useful.

A language-model agent without a play mechanism has no equivalent first week. It cannot pull the repository and poke at it in a sandbox for a while without being asked to. It cannot run tests with modified inputs on its own initiative and keep what they produced. The observations it would have made by poking around never exist. When a real task arrives, it has exactly the context the prompt provides, the documentation it can retrieve, and the general priors from training. The specific facts about this environment, the ones the human colleague would have accumulated, are not available.

The failure mode is subtle. The agent does not report that it lacks familiarity. It produces an answer that draws on general priors and retrieved documentation, and the answer is often plausible. It is just wrong in the specific ways that familiarity would have caught:

• The test that is flaky on this codebase gets run, and its result is taken at face value.
• The config value that is load-bearing despite appearing optional gets changed without comment.
• The service that fails slowly gets called in a pattern that assumes fast failure.

Each individual mistake looks like a reasonable answer. The pattern across them is the signature of a system that never had the first week.

Giving the agent play access to the environment is necessary for closing this gap but not sufficient. The agent also needs:

• the disposition to use the access without being prompted;
• a way to retain what it observed; and
• the ability to draw on those observations when real work arrives.

Access alone produces a system that could have played but did not. Play has to be a standing mode the agent enters in appropriate environments rather than a feature it exercises under instruction.

The facts outside the corpus

Every AI system has a set of facts it can access and a set it cannot. For systems restricted to reading, the accessible set is what was recorded. Everything that was never written down sits outside.

The question for any deployed system is whether the work it is being asked to do depends on facts in the outside set. For many deployments, it does. The codebase the system is working in is specific. The environment is specific. The behaviors that matter are specific. General priors and public documentation go only so far.

A system that can discover facts through interaction has access to things reading cannot reach. A system that cannot has exactly what the corpus gave it. The difference shows up as the difference between a colleague who spent the first week becoming familiar with the environment and one who did not.

References

• Bellemare, M. G., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., & Munos, R. (2016). Unifying count-based exploration and intrinsic motivation. Advances in Neural Information Processing Systems 29.
• Pathak, D., Agrawal, P., Efros, A. A., & Darrell, T. (2017). Curiosity-driven exploration by self-supervised prediction. Proceedings of the 34th International Conference on Machine Learning.
• Togelius, J. (2024). Artificial General Intelligence. MIT Press Essential Knowledge series. Identifies open-ended learning in virtual environments as one of the two main technical approaches to more general AI, alongside foundation models.
• Wang, R., Lehman, J., Clune, J., & Stanley, K. O. (2019). POET: Open-ended coevolution of environments and their optimized solutions. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’19). Introduces the Paired Open-Ended Trailblazer, which generates its own environments and solves them, producing a self-expanding curriculum without a human designing it.

Originally published on Medium.