4 Answers

Things we may not find appealing have a whole new attraction to puppies, particularly when they are young, playful and curious

The puppy probably tries to get inside your trash because it smell food remains inside of it. Give the puppy food and remove the remains and it probably would stop.

Because it smells so interesting. Puppies, like human toddlers are learning. He smells things that he needs to put an image in his mind to. Dogs have very acute sniffers. They can smell stuff that we can not. He is smelling a big mix of orders and needs to learn how to distinguish each odor separately  This is why he's into the trash. Plus to him, there just might be something good to eat in there ;) Lastly, I think it's just plain fun for him. 

His motivations are different than ours, he is more scent driven, thus this interesting and stimulating smell intrigues him.  Remember, he is not trained to exist in our world yet, and his evolutionary behaviors have to be over-ruled through training.  He is wired to survive, and sometimes as a hunter, other times as a scavenger.  So understand him, do not punish him, but do remove this temptation from within his reach!