Why is the server using too much charge of the memory, whereas the server has been freshly installed !?
Don’t worry, this is Linux that uses the real memory to store cache files in order to be faster.
It will always let a certain margin with the swap file (virtual memory) just in case and will quickly clear the hard memory when your sever will really need it. This is how Linux is working.
What is important is the total available memory.
You can still use these commands to clear the cache :
sync; echo 1 > /proc/sys/vm/drop_caches
sync; echo 2 > /proc/sys/vm/drop_caches
sync; echo 3 > /proc/sys/vm/drop_caches