We did debate whether to fill the pallet walls with some sort of insulation like polystyrene packing beads or even gun foam but it never happened.
Without a doubt the way @Olderisbetter has gone with Celotex is the way to go for most builds. I'll be hopefully doing similar when I do a passive esque 30m2 garden room. Saying that a lot of passive self builders I know go for blown cellulose because of the better decrement delay. Chap I know near Canterbury only uses about 1.2kW a day for space heating the entire house. Mind his walls are 400mm thick filled with Icynene. Expanding foam that comes on a truck with two bfo tanks.
I once use those polystyrene beads as insulation for an outside tap feed in a shed, boxed in, unfortunately the pipe leaked, and the beads went all slimey.