A curious agent acts so as to optimize its learning about itself and its environment, without external
supervision. We present a model of hierarchical curiosity loops for such an autonomous active learning
agent, whereby each loop selects the optimal action that maximizes the agent’s learning of sensorymotor
correlations. The model is based on rewarding the learner’s prediction errors in an actor-critic
reinforcement learning (RL) paradigm. Hierarchy is achieved by utilizing previously learned motorsensory
mapping, which enables the learning of other mappings, thus increasing the extent and diversity
of knowledge and skills. We demonstrate the relevance of this architecture to active sensing using
the well-studied vibrissae (whiskers) system, where rodents acquire sensory information by virtue of
repeated whisker movements. We show that hierarchical curiosity loops starting from optimally learning
the internal models of whisker motion and then extending to object localization result in free-air whisking
and object palpation, respectively.