Generally the term is used when the database access mechanism gets along with the application, as opposed to having another application that your own communicates to access the database.
The best known embedded database is SQLite (and it is also the most widely used database considering all types of DB).
In this case the product allows it to work as well or as a server. Each has its advantage.
For simple things that will run only on that machine it is usually best to use it on the application. Just do not know how good the H2 is in this mode, it is common to have some limitations, some products have too many limitations. But it tends to be faster and simpler to deploy and care for. Almost all the websites in the world (I do not speak of web applications, many can, but not much) can be well served by an embedded database, even because the overwhelming majority of them have little access and little interaction. A little redesigned, probably this site (the English version itself, one of the most accessed sites in the world, but practically an application) would run with SQLite or some embedded DB, with advantages and disadvantages.
If you have multiple instances of applications wanting to access it at the same time, especially on different machines the server passes be better, or even needed.