Dude, I'm 39 and I just started training in the past few years. You get no sympathy from me about not starting young!
Seriously, don't worry if other guys in the gym are a lot better than you at first. Don't view training as a competition yet -- sure, you may compete down the road, but at first just concentrate on learning and developing your conditioning and skills each day. Your training program sounds good, and you will find lots of additional helpful ideas in this forum.
Look at it this way -- most of the guys I train with, in my newbie BJJ class, are in their early twenties. If you start training regularly now, you will be seriously badass when you are their age (not to mention my age...)