Integrated Calendar

Recent text-to-image diffusion models are able to generate convincing results of unprecedented quality. However, it is nearly impossible to control the shapes of different regions/objects or their layout in a fine-grained fashion. Previous attempts to provide such controls were hindered by their reliance on a fixed set of labels. To this end, we present SpaText - a new method for text-to-image generation using open-vocabulary scene control. In addition to a global text prompt that describes the entire scene, the user provides a segmentation map where each region of interest is annotated by a free-form natural language description. Due to lack of large-scale datasets that have a detailed textual description for each region in the image, we choose to leverage the current large-scale text-to-image datasets and base our approach on a novel CLIP-based spatio-textual representation, and show its effectiveness on two state-of-the-art diffusion models: pixel-based and latent-based. In addition, we show how to extend the classifier-free guidance method in diffusion models to the multi-conditional case and present an alternative accelerated inference algorithm. Finally, we offer several automatic evaluation metrics and use them, in addition to FID scores and a user study, to evaluate our method and show that it achieves state-of-the-art results on image generation with free-form textual scene control.

The global oceanic overturning circulation and the transport of heat and dissolved gases are strongly controlled by upper ocean turbulent mixing that is driven by the breaking of internal gravity waves (IWs). Understanding the life cycle of oceanic IWs, from generation to dissipation, is therefore crucial for improving the representation of ocean mixing in climate models, which do not resolve the IW field.  Oceanic IWs are observed to have a continuous energy distribution across spatial and temporal scales – an internal wave continuum – despite being forced primarily at near-inertial and tidal frequencies at large scales. The formation of the IW continuum and the associated energy transfer to dissipative scales have been traditionally attributed to wave-wave interactions and to Doppler shifting of wave frequencies by currents. Here, we provide evidence from realistic numerical simulations that oceanic eddies rapidly diffuse storm-forced wave energy across spatiotemporal scales, thereby playing a dominant role in the formation of the IW continuum and the corresponding spatiotemporal distribution of energy dissipation. We further demonstrate that winds can play an important role in damping oceanic IWs through current feedback.  This results in a substantial reduction in wind power input at near inertial frequencies and a net energy sink for internal tides.

TBA...

Introduction: Bianca Jones Marlin, Ph.D. is a neuroscientist and Herbert and Florence Irving Assistant Professor of Cell Research at the Zuckerman Institute at Columbia University in New York City. Her research investigates how organisms unlock innate behaviors at appropriate times, and how learned information is passed to subsequent generations via transgenerational epigenetic inheritance. Dr. Marlin combines neural imaging, behavior, and molecular genetics to uncover how learned behavior in the parent can become innate behavior in the offspring— work that promises to make a profound impact on societal brain health, mental well-being, and parenting. For more information about Dr. Marlin, visit www.biancajonesmarlin.com

Noise plays a central role in many machine learning algorithms

TBA..