The systematic isotopic difference between refractory inclusions and chondrules, particularly for oxygen, has long indicated an isotopic evolution of the solar protoplanetary disk. However, it remains underconstrained whether such evolution continued during chondrule formation. Intrigued by past reports of the size-dependent oxygen isotopic compositions of chondrules in ordinary chondrites (OC), we analyzed type I olivine-rich chondrules of various sizes in two LL3 chondrites. Although most chondrules show positive Δ$^{17}$O values comparable to bulk ordinary chondrites, a population of smaller (less than about 0.1 mm2 in cross-section), including many isolated olivine grains (sensu lato), are 16O-enriched (with Δ$^{17}$O values down to −13.2 ‰). Literature data allow the same observation for R chondrites. All sub-TFL type I chondrules (i.e., Δ$^{17}$O < 0) chondrules have Mg# > 97 mol% while the supra-TFL type I chondrule olivines extend to the formal boundary with type II chondrules (i.e., Mg# = 90 mol%). The sub-TFL chondrules are likely genetically related to isotopically similar aluminum-rich chondrules described in the literature. They therefore must have formed earlier than most OC and R chondrules when the inner disk was still sub-TFL. This interpretation is supported by the presence of similarly $^{16}$O-rich relict grains in supra-TFL OC and R chondrules that must be remains of this incompletely recycled precursor material. The non-carbonaceous reservoir was thus still evolving isotopically towards 16O-poorer composition when chondrule formation began, whether by mixing with outer disk material, late accretion streamers and/or an increase in the solid/gas ratio due to magnetothermal disk winds.