Unlike the standard policy, our policy is not affected by the suddenly permuted inputs. Left: The ordering of the ant’s 28 observations are randomly shuffled every 200 time-steps. Permutation invariant reinforcement learning agents adapting to sensory substitutions. Our experiments show that such agents are robust to observations that contain additional redundant or noisy information, and to observations that are corrupt and incomplete. In “ The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning”, a spotlight paper at NeurIPS 2021, we explore permutation invariant neural network agents, which require each of their sensory neurons (receptors that receive sensory inputs from the environment) to figure out the meaning and context of its input signal, rather than explicitly assuming a fixed meaning. In popular RL benchmark tasks (e.g., Ant or Cart-pole), an agent trained using current RL algorithms will fail if its sensory inputs are changed or if the agent is fed additional noisy inputs that are unrelated to the task at hand. They expect fixed-size inputs and assume that each element of the input carries a precise meaning, such as the pixel intensity at a specified location, or state information, like position or velocity. For instance, most reinforcement learning (RL) agents require their inputs to be in a pre-specified format, or else they will fail. In contrast, most neural networks are not able to adapt to sensory substitutions at all. Right: “Upside down goggles” initially conceived by Erismann and Kohler in 1931. Left: Tongue Display Unit ( Maris and Bach-y-Rita, 2001 Image: Kaczmarek, 2011). While difficult adaptations - such as adjusting to seeing things upside-down, learning to ride a “backwards” bicycle, or learning to “see” by interpreting visual information emitted from a grid of electrodes placed on one’s tongue - require anywhere from weeks, months or even years to attain mastery, people are able to eventually adjust to sensory substitutions.Įxamples of Sensory Substitution. This adaptive ability, called sensory substitution, is a phenomenon well-known to neuroscience. People have the amazing ability to use one sensory modality (e.g., touch) to supply environmental information normally gathered by another sense (e.g., vision). Posted by David Ha, Staff Research Scientist and Yujin Tang, Research Software Engineer, Google Research, Tokyo
0 Comments
Leave a Reply. |