How Young Is Too Young To Start Playing Sports?

My wife and I are looking to introduce our son to the sport of soccer. He is three years old but we fear he may be too young? We want to make sure he is able to understand the rules of the game and have the attention span to focus through an entire game.

As a parent I just want what is best for my son. I want him to love soccer and keep him active but I just don't want to introduce him to the sport too young.

What do you think? Should my wife and I go ahead and sign him up or should we wait until he is a bit older?