I attended a Meetup once and one of the topics was world building. The level of detail that went into this one person's world was incredible. His challenge was the story. For me, I had the story but the world was a blank slate. Of course, I knew how to get to the world (via the moon shadows) and I knew the people of that world lived by the sword, but that is all I knew.