If you are part of a large alliance, then your alliance will probably come under significant pressure to make you stop. Whether an individual player is large or small, if the player is part of an alliance, that means a player hitting an individual non-aligned player is basically picking on someone smaller than themselves. No matter how much people will protest that it is 1v1 play, it always reflects on the alliance, because the alliance would protect the player if things go south.
If an unaligned player attacks another player of similar size who is not aligned, then the player who was attacked may call in his/her Illy friends and/or may simply choose to join a large alliance.
But aside from those general observations of past behavior, I'd say that anything might happen at any time. I personally think that people should exercise individual freedom in how they respond to such situations, which would tend to make it a bit unpredictable as to what might happen.
If worse comes to worse and your cities get sieged to the ground, you can rebuild or start another account. Building cities is fun!