As far as I know, there is no difference between the commercial and other versions in terms of the functionality, the only difference is the licensing.

If you have version 12, both the new compiler and the Database Integration project are parts of it. Finding where they are located is not hard. For example, for Database Integration:

<< Databases`
Lookup[PacletInformation["Databases"], "Location"]

However, both code bases are rather involved, and reading the code is probably not an easy task for someone not familiar well with the internal architecture of the projects.

I am not aware of many resources on meta-programming in WL, particularly on the level of official documentation. You can search for some posts on Mathematica at Stack Exchange.

For dealing specifically with languages and grammars, and actually also for some aspects of metaprogramming, look for posts of Anton Antonov, here on this site, on M SE, and on his blog. He has built a number of tools in this area, and made a number of posts explaining those. There is also a nice chapter on implementing languages in WL, in the book "An introduction to programming with Mathematica" by Wellin, Gaylord and Kamin. They show how to implement a simple graphical language.

It is possible, yes. As far as I know, a large part (if not all) of the new WL compiler has been written in WL. Also, we wrote the subset-of-WL to SQL compiler mostly in WL, for the new Database Integration project for V12.

I think that symbolic trees representation and transformations are among strong sides of WL. Also, the frontend makes it easy to visualize (parts of) AST, and other relevant data structures. Pattern matching can be a natural tool to work with grammars. So if you are looking at ways to prototype and visualize, then WL is among the best tools available, I think.

One thing you should know is that WL is strongly geared towards immutability, so for best experience, all your data structures (AST, etc.) better be immutable, and stateless. We have used this approach in Database Integration project with very good results, we have zero state in our entire compilation pipeline. But this style of programming needs some time to get used to, unless you are coming from some pure functional languages like Haskell.

Hans,

Metaprogramming is a technique which, as a concept, is rather language-agnostic, as long as the language supports the necessary abstractions to make it easy or possible. For a general description, you can start with the Wikipedia article. In WL, there are a few markers which indicate the use of metaprogramming. To give you a few examples, defining your own scoping construct or inspecting / destructuring some code (written program) will be metaprogramming. Very often, functions which use metaprogramming have one of the Hold - attributes, or otherwise use what is called non-standard evaluation, since they need to prevent some pieces of code from premature evaluation. So, while perhaps not all functions with Hold - attributes can be considered as using metaprogramming, many can.

To give simple examples, some "usual" function like

f[x_]:=x^2

is obviously not using metaprogramming, while, for example, a function which inspects some code and returns a list of all symbols (wrapped in HoldComplete to prevent their possible evaluation), from which that code is built:

ClearAll[getHeldSymbols];
SetAttributes[getHeldSymbols, HoldAllComplete];
getHeldSymbols[expr_]:=Cases[Unevaluated[expr], s_Symbol :> HoldComplete[s], {0, Infinity}, Heads -> True]

does use metaprogramming.

Possibly useful for metaprogramming: VisX software for Wolfram Language.