Well, it would create quite a problem for you when you turn the compost, or use it. Most honeybees are pretty peaceful unless they are defending their hive.
Honeybees usually look for a small hole as the entrance with a larger volume of enclosed space. They are not noted as diggers, they almost never hollow out their own nest space. At most they may do a small amount of chewing, such as removing tar paper inside the walls of a house or other building.
Does your compost pile have such a space that they would have moved into?
Maybe they are some other sort of bee that is more willing to dig?
Just as an example, I know they look very different, but Bumble bees would be much more likely to move in to a setting like that. Around here I have seen them in a pile of grass that a guy mowed and added to every week. I have also seen them under a deck, in many years accumulation of leaf litter.