Casimir force between perfectly reflecting nonreciprocal plates and its duality variance