Sense and Logics would seem to indicate that to save a penny, you need to earn it first, otherwise you won't have a penny to save. But as we have seen time and again, to investigate fully it is necessary to put Science and Googles on a chart, then look at it with our eyes.
Here we see that ever since the discovery of the penny in the late 1700s it has been saved, as shown by the blue line.
Oddly, we can also see that penny saving peaked around 1815, at around the same time that penny earning, shown by the red line, had only just begun.
Even more mysteriously, we can see that in the whole of recorded history, far more penny has been saved than earned!
Now we must ask ourselves, in a deep scientist type voice, "How can that much penny be saved when so little penny has been earned? Hmm?". Thankfully this is exactly the sort of question that Science and Googles can answer.
As can be seen, there are numerous sources of penny to consider. These are: penny earned; penny stolen; penny found; pennies from heaven; and the proverbial bad penny.
When all of these penny sources are added together, they probably equal the amount of penny saved (apart from a few penny losts, which of course do not show up on the chart because they are lost).
However, this still leaves the question of how so much penny can be saved back in 1815 before it's even been earned, stolen, found, fallen from heaven, or turned up. And of course, the only Logical answer is, by the use of a Time Machine!
In conclusion then: yes a penny saved is a penny earned, or gained by some other means, as noted above, but then transported back in time to 1815 by means of a Time Machine.