Children's Books discussion

Hi,

I am trying to remember the name of a children's book. I thought the title was The Enchanted Castle or The Magic Castle and published in the 50s or 60s. It's not the E. Nesbit one.

Plot:
A little girl chases after her dog and meets a man who says he is waiting. He's waited for years to join his wife and children. He's a fairy or something and when he met his wife she was mortal. The deal was they could live happily but there would come a day when she would go on ahead and he had to to wait to join her.

As he's waiting he tells the little girl about his wife, his son who fell for a girl that was enchanted by goblins or something. I think his daughter fell in love with a mortal.

At the end he disappears and goes to the other land. His wife and the dog had the same name, something like Floramund??

Any help would be appreciated!

Shadow Castle