The Virasoro constraints are one of the major open problems in enumerative geometry, originally in Gromov-Witten theory, but more recently also for moduli spaces of sheaves or representations of quivers. The latter have connections to vertex algebras and wall-crossing. In this talk, I will explain how this story looks like in the simplest possible example: Grassmannians. It turns out that such structures, in this context, become related to well-known combinatorial and representation-theoretic aspects of the algebra of symmetric functions, such as Schur polynomials and their vertex operator realizations. The talk is based on joint work with Woonam Lim.