Not possible to set in Settings. And also not usual in plan drawings.
If you desperately want or need it you can add the unit in the Tool Options bar above the drawing area (changing it's contents according to the command in action) by typing <>mm (or whatever unit):
Or edit an existing dimension with Properties in the same way.