I enjoy all the analogies in the topic. Haha.
In addition to what everyone else said, I also think teams deem the kicking and punting duties as special enough that you need two guys to do it. I'm sure Andy Lee, if he spent the time, could be a decent-enough kicker... but let's say it's the Super Bowl and we have to kick a field goal to win it... Would you rather have Andy Lee, who switches between kicking and punting? Or would you rather have a guy who spends all of his time kicking?
I also don't know how tired a guy can get. If Andy Lee spent all of his time booting punts all day, and then had to be in the position to kick a 50 yard field goal later on, I have no idea if he'll have the leg to do it.