In many, many cases this appears to be the case, sadly. (although hearing someone try to scare someone about postpartum hemorrhage after 2 pregnancies is a new one for me!)
No I want you guys to have dairy still if you like it. No benefit to switching over to almond milk that we're aware of, and many people may lower their fertility too far if they go fully vegan.