DeepMind adds to their earlier work on specification gaming, by noting a new AI failure mode, namely, goal misgeneralization, where an AI model’s capabilities successfully generalize to a new environment, but the goal itself does not, leading to undesirable behavior.
Researchers have found scaling laws for certain reinforcement learning techniques, enabling them to determine how big a neural net you need for optimal performance of your RL model. Not only did the researchers find such scaling laws, but also applied them to AlphaGo Zero and AlphaZero–two impressive RL models by DeepMind–finding that these neural nets were too small to reach optimal performance and thus, could “achieve better performance with larger neural nets.” Bad news for humans hoping to make a comeback in Chess or Go.
Pretraining on image datasets is still the standard practice, however videos also contain rich representations of the world, and yet video pretraining has not been particularly successful thus far. This new paper closes the gap for ImageNet pretraining, suggesting that video pretraining could become the new dominant paradigm in the future.
Check out Phenaki, a text-to-video model, but unlike Make-A-Video and Imagen, it can generate videos of up to multiple minutes long.
