The Protestant Reformation answered that question with a resounding 'Yes!' Scripture reveals that the children of believers are part of God's visible church and covenant people, and therefore ought to receive the sign of membership in God's covenant community. Pastors Michael Brown of Christ URC and Chris Gordon of Escondido URC take up this question on a recent program of Abounding Grace Radio. You can listen to the program here.
Pastor Brown also wrote a short article explaining the biblical and Reformed understanding of baptism, which you can read here.