The government generally spends more than it raises in tax. To fill this gap it borrows money, but that has to be paid back - with interest. The government gets most of its income from taxes.