Central banks have emerged over the past four centuries when mankind moved from a system of gold or silver backed currencies to private issuers to fiat money. The first central bank in the world was the Swedish Riksbank, founded in 1668.
Scottish businessman William Paterson founded the Bank of England in 1694 on request of the British government to finance a war.
The First Bank of the USA was founded in 1791 and had a 20-year charter. It was however revived again in 1816 and gave birth to the Second Bank of the United States. This desperate move to stabilize the currency by US president James Madison was later revoked by US president Andrew Jackson who withdrew the bank's Federal Charter in 1836. In 1841 the Second Bank of the United States ceased all operations.
The history of central banking came alive again in 1913 with the constitutionally disputed foundation of the Federal Reserve.
As nearly all currencies in the world have transformed into fiat money over the past 4 centuries, i.e. the notes mandated to be used by government fiat, all countries have some sort of central bank that is responsible for keeping inflation low and provide a money supply that does not overshoot economic growth too much. Other tasks vary widely from country to country.
Central banks primarily purchase short-term debt issued by the governments of the countries in which they serve. Sometimes they've been used as a political piggy bank with the central banks buying riskier investments causing huge losses and inflation if the bank's government doesn't bail them out since the bank has no assets to sell to counter inflation, leaving to many banknotes in circulation.