neutral faction took over my outpost and shard

so, as a new player, I need to ask how this mechanic works?  I had an outpost built next to a shard and had used up a logistics point to tap (build) on the shard for its magic.  About 20 turns later a neutral faction plops down a settlement just outside of my outpost area with the shard.  A few turns later I noticed that my entire area along with my outpost now belonged to the neutral faction.  Is that supposed to happen?