Metasurfaces offer a unique playground to tailor the electromagnetic field at subwavelength scale to control polarization, wavefront, and nonlinear processes. Tunability of the optical response of these structures is challenging due to the nanoscale size of their constitutive elements. A long-sought solution to achieve tunability at the nanoscale is all-optical modulation by exploiting the ultrafast nonlinear response of materials. However, the nonlinear response of materials is inherently very weak, and, therefore, requires optical excitations with large values of fluence. We show that by properly tuning the equilibrium optical response of a nonlocal metasurface, it is possible to achieve sizable variation of the photoinduced out-of-equilibrium optical response on the picosecond timescale employing fluences smaller than 250 μJ / cm², which is 1 order of magnitude lower than previous studies with comparable reflectivity variations in silicon platforms. Our results pave the way to fast devices with large modulation amplitude.