The Solar System is the area of influence around a star such as the Sun. Other Stars such as Fomalhaut in Piscis Austrinus and Pollux in Gemini will have their own Solar Systems. There is nothing to say that other Solar Systems don't have equivalent asteroids belt or a Kuiper Belt or their own Oort Cloud.
"The Solar System" refers to our solar system that contains the Sun and the Earth. When we use without capitals, it is a type of system in the same way planet is.
The below is an artists impression of the Kepler 22B Solar System, a multi-planetary system that lies about 1,200 light years away in the constellations of Lyra. The Star has a number of planets which lie within the Goldilocks Zone where the area of space is not too hot or not too cold for life to exist on the planet. Sadly for our Solar System, the only planet in the Goldilocks Zone and has life is our own planet Earth. Mars once might have supported life but if there is any life on the planet, it will be micro-organisms life forms.
Our Solar System contains the eight planets and untold asteroids in the Kuiper Belt and Asteroid Belts. The Solar System also includes if it exists, the Oort Cloud but its only a theory and there's no hard evidence it exists yet. The furthest confirmed object that is part of the Solar System is the Kuiper Belt Object of Sedna. There could be further objects out there but they have not yet been discovered. The below image from National Aeronautics and Space Administration (N.A.S.A.) shows the current view of the Solar System.
In the picture, the planets are not drawn to size, the Earth would be smaller than a pixel in comparison to the Sun. The Uranus is further from Saturn (9.60 AU) as Saturn is from Earth (8.54 AU). 1AU is the distance from the Sun to the Earth. 1
Whilst we have eight planets and three dwarf planets (Pluto, Ceres and Eris), there could still be a giant planet outside the orbit of Neptune still waiting to be discovered. The new horizons mission to pluto whilst focused primarily on Pluto may end up discovering the new planet either whilst travelling to Pluto or afterwards.
We have already sent probes out past the edge of our Solar System, past the Heliopause, the point at which the Sun's radiation meets Interstellar radiation.
There is a theory that rocky planets such as Mercury, Venus, Earth and Mars form close to the Star and giant ones form on the outskirts. They also say that Gas Giants such as Jupiter and Saturn gravitate their way into closer to the Sun but this has not happened yet but may in the far future.
The universe is 13.8 Billion years old give or take a few thousands or even millions of years but our Solar System would be about is 4.6 Billion Years ago. There are solar systems that older and some that are newer than ours. It was announced as part of the Kepler project, that they had found a star system that is 11.2 Billion Years old which is more than double ours. The Solar System is in the constellation of Lyre.
Back to our Solar System, the system would have been a large cloud of dust and gas and then something would have caused the gas to collapse with the Sun forming at it centre and the planets forming round the outside. In the video suggests an older star collapsed and turned into a supernova. The Supernova would be the kick that the cloud need to begin to coalesce and form the planets and everything in the solar system. The solar system will last until the Sun runs out of energy and it too supernovas in about five billion years time.
The Solar System is the smallest of the group, as mentioned above, it is the area of influence around a star such as our own star, the Sun. A solar system may have multiple stars within itself such as in the case of Regulus or Algieba. There will be a centre of mass which will be the centre of the solar system. The centre might not be in the centre of a star if its a binary star system. If the two stars are of near equal power and size then the centre of mass could be in between both the stars because neither star has the strength to cause the other star to orbit round it. The Solar System will cover all the planets, asteroids and as far out as any possible equivalent Oort Cloud. To some people, the Solar System only goes as far until the Heliosphere ends where the Solar winds meet the intergalactic winds. For me the solar system ends at the Oort Cloud, agree to disagree.
A galaxy is a large collection of stars, nebulas, solar systems that are bound to a supermassive black hole such as the one at the centre of our Milky Way. When I say large collection, I mean millions or even trillions of stars. Our Milky Way is 100,000 Light Years across but its a small galaxy compared to say Andromeda Galaxy at 200,000 Light Years or the largest Galaxy so far discovered IC 1101 radius is estimated at being over 2 million light years.
The Universe is everything, all the galaxies, all the solar systems, there is no centre of the Universe which everything orbits rounds such as in the case of Solar Systems and Galaxies. The currently widely accepted school of thought for the Universe is that its spreading out and therefore there's no centre of mass. Its estimated at being over 96 billion light years across and spreading out further.
The only thing bigger is the hypothetical multiverse and parallel universes / worlds which is a collection of universes, with universes in bubbles spread over an unimaginable area of space or different realms (many worlds). We don't know if it exists and universe move.