The drawing unit parameter is just to give drawing units a meaning.
So changing the unit doesn't change the drawing.
When the dimension shows 4940, means the drawing was created with mm in mind.
So to change this you can either scale the whole drawing or change the length factor in Dimensions tab of Drawing Preferences.
The easier way is the length factor. Set it to 0.001 and all dimension values will change to meter.
All settings in Dimensions tab are set for mm. So when the drawing would be scaled to meter, all dimensions will become gigantic.
That means, when the drawing would be in meter, all Dimension settings there must be scaled, e.g. arrow size should then be 0.0025, not 2.5.
Except the length factor, which is is correct then at 1.0.