If, say, I need an explosive charge of around 5kg in a HE projectile, what is the most compact howitzer and/or direct-fire-capable mortar I can use?
I don't quite understand what you want to accomplish.
I didn't find good mortar shell specifications - but generally they seem to have pretty small amounts of explosive, being optimized for generating a lot of horizontally flying shrapnel to wipe out soft targets.
You can launch shells of different weights and lengths out of the same mortar tube. Heavier shells just have less range.
A mortar, with base plate on the ground or light(ish) vehicle, tends to favour shooting at elevations above 45 degrees. That gives relatively long flight times. If you want to go below 45 degrees, you are probably better off with something else. If you are close enough to the target, to see it through your mortar's aiming device, you are in trouble.