The idea that yeast is a common allergen is a misinterpretation of some blood tests that often get run in inflammatory bowel diseases (IBD) like Crohn's, ulcerative colitis (UC), and celiac. When someone suffers from these diseases, they frequently also have compromised integrity of the intestinal lining. This allows more potential antigens to enter the blood, which will stimulate an immune reaction.
Anti-Saccharomyces cerevisiae antibodies (ASCA) can be used as a diagnostic tool for IBD, because people with IBD can be expected to frequently have compromised linings compared to healthy controls. Thus, people with IBD tend to produce ASCA when exposed to dietary yeast (which is almost always *Saccharomyces cerevisiae*, whether it's used to make beer, wine, bread, or nutritional yeast).
It's unclear how much of a problem these antibodies actually are for someone with an IBD, and it appears to depend on the specific IBD the person suffers from. Crohn's disease appears to have the most frequent reaction with yeast, so for someone with Crohn's it may be worth trying to remove yeast-containing foods/beverages to see if there is any improvement. For celiac and UC, it doesn't appear as if the yeast itself is much of a problem in the majority of cases.
If you do test positive for ASCA, the biggest thing I would take from it is that chances are good your intestinal lining is compromised, which is really the bigger issue. Our immune system is designed to get rid of foreign particles, so I would argue against a general immune reaction being a sign of an "allergy". An allergy is an OVERreaction of the immune system, not just any reaction. Regardless of whether you have a reaction to a particular food, if your gut lining is compromised, you'll continue to have reactions large or small as foreign particles inevitably make their way into your blood!
So, unless you have an IBD, I wouldn't worry about yeast being an allergen--I could find only one case study of a boy who suffered a true yeast allergy, and otherwise all reactions to yeast occurred in individuals with an IBD.
Yeast doesn't *cause *any IBDs, no more than gluten *causes *celiac disease--these diseases have a complex and not fully-understood etiology, but it likely involves genetics to a great extent. But, just as gluten aggravates celiac disease, yeast may aggravate certain other IBDs, especially Crohn's.