Silver is used for conquests.
For every castle you own you need 1000 silver (for each fortress 5000) to conquer another castle.
At the keep silver can be exchanged for wood, stone and ore. If the keeps level is higher the exchange rate is better.