Why yes, you can! In fact, these years are such a wonderful time to begin homeschooling because you can establish good habits in your younger children that will serve as great helps when they are older.
Before we talk about that though, let's look at your state laws. Remember, homeschooling is legal in all 50 states, but each state has it's own set of regulations homeschool families must follow. For the purpose of this post, I am talking about those years before compulsory schooling is mandated by your state. That age varies from state to state. In Texas, children must attend school from age 6-19, or until they graduate or earn a GED. However, if they have been enrolled in school for Pre-Kindergarten, Kindergarten, or First Grade, they must continue school that year, even if they are not yet 6. So, you will want to check your state's compulsory age for school and any little bits like that you may need to know.
If your child is not yet of compulsory school age (and meets all the other requirements), you can still homeschool them! It may not be "official", in that you aren't legally required to do schoolwork with them, but we aren't only teaching our children to fulfill a law, right? We want to give our babies the best start possible, whether you intend to homeschool them through 12th grade, or enroll them in public or private school when they are older. No matter what the future holds, you can have a great homeschool experience with your young children.
I have had the privilege to know some really excellent Pre-K and Kindergarten teachers in my life. I have to admit, I had a slight pang of sadness that my girls weren't going to get to experience a superb classroom environment like I knew was out there. If only we could hand-pick the experiences our children have! My concerns were soon laid to rest when I found out how much fun there is to be had in homeschooling these younger years! They are honestly some of my favorites and I will do what I can to make sure your younger year homeschooling is awesome as well!