Struggling Writers discussion

I have a very important question about this. Right now there is just a book of poetry. I also have several children's stories that I want to put pictures to. However, there is some fiction that would be considered very controversial and maybe taboo. LOL What would the risk be of losing my following if it built up with the poetry and children's stories? Should I write under the name I am using here or pick a different pen name? Does it even matter?