Make a slurry by mixing cornstarch with 3 Tbsp almond milk. Set aside.
Heat the remaining almond milk and sugar in a saucepan over medium heat, stirring regularly.
Add the slurry to the hot milk, stirring constantly until thickened. Test by dragging a spatula through; it should take about a second to fill back in.
Remove from heat, cover with plastic wrap touching the surface, and let cool.
Blend peaches and lemon juice into a smooth puree. If too thick, add 1-2 Tbsp water.
Pour 2 Tbsp peach puree into each popsicle mold, followed by 2 Tbsp milk pudding. Alternate layers until molds are full, leaving some space for expansion.
Tap molds on the table to remove air bubbles. Optionally, create swirls with a butter knife.
Insert sticks, leaving about 1.5 inches exposed.
Freeze popsicles for at least 8 hours.
To unmold, dip tips of the mold in hot water for 10 seconds, then pull out using the sticks.