Recent advances in language and vision models have helped make great progress in creating robotic systems that can follow instructions from text descriptions or images. However, there are limits to ...