mill-pdef

Generates scodec codecs from .pdef binary layout definitions.

This is the Mill port of sbt-pdef-gen, repackaged under nscn.polaris2.pdefgen. A .pdef describes a flat, fixed-size binary record — the format Titanfall 2 uses for its persistent player data — and the plugin turns each one into a Scala source file holding a case class per struct, an enum per enum block, and a scodec Codec for every one of them.

Layout

Module	Artifact	What it holds
core	nscn.polaris2::mill-pdef-core	The runtime codecs the generated sources call into
plugin	nscn.polaris2::mill-pdef-plugin_mill1	PdefModule, the pdef parser, and the code printer
example	—	An end-to-end build that decodes a real default.pdata

core is a normal library and has to be on the consumer's classpath; PdefModule adds it to mvnDeps automatically.

Using it

//| mill-jvm-version: system
//| mvnDeps:
//| - "nscn.polaris2::mill-pdef-plugin_mill1:0.0.1"

package build
import mill.*, scalalib.*
import nscn.polaris2.pdefgen.PdefModule

object app extends ScalaModule, PdefModule {
  // Must be >= the Scala version `core` is built with.
  def scalaVersion = "3.9.0"

  // .proto-style convention: `.pdef` files live in `app/pdef` by default.
}

app/pdef/Pdata231.pdef then yields case class Pdata231 and codec_Pdata231 in package nscn.polaris2.pdefgen.generated, wired into generatedSources so a plain ./mill app.compile picks them up. Every .pdef in the module is checked before anything is generated.

Settings

Task	Default	Purpose
pdefSources	<module>/pdef	Directories searched for .pdef files
pdefPackageName	nscn.polaris2.pdefgen.generated	Package the generated sources are emitted into
pdefVersion	PdefModule.defaultPdefVersion	Version of mill-pdef-core added to mvnDeps
pdefPackageDefinitionsAsResources	true	Also ship the .pdef files as resources
pdefFiles	derived	Every .pdef found, in a stable order
pdefGenerate	derived	Runs the generator, returns the output directory

The pdef language

// line comments run to the end of the line

int      initializedVersion          // a scalar field
string{16} lastFDTitanRef            // 16 BYTES, NUL padded
int      xp_match[20]                // fixed length array

$ENUM_START titanClasses             // enum block
    ion
    scorch
$ENUM_END

$STRUCT_START recentUnlock           // struct block
    int refGuid
    int count
$STRUCT_END

recentUnlock recentUnlocks[10]       // array of a struct
int          titanXP[titanClasses]   // array indexed by an enum

Types map across as:

pdef	Scala	codec	width
int	Int	int32L	4 bytes, little-endian
bool	Boolean	pdefBool	1 byte, 0x00 / 0x01
float	Float	floatL	4 bytes, little-endian
string{n}	String	pdefString(n)	n bytes, NUL padded
T x[n]	Vector[T]	vectorOfN(provide(n), …)	n × T
T x[E]	Map[E, T]	pdefEnumMap(E.values, …)	one T per variant of E
$STRUCT_START	case class	codec_<name>	sum of its fields
$ENUM_START	enum	pdefEnum(<name>.values)	1 byte ordinal

Two consequences worth spelling out:

• Declaration order is the wire format. The fields of a pdef are laid out back to back with no tags, lengths, or padding, so reordering them silently changes what the codec decodes. Both the parser and the printer preserve source order for exactly this reason.
• An enum's ordinal is one byte, so a $ENUM_START block may not exceed 256 cases. The plugin refuses to generate anything larger. Decoding a byte that is no case's ordinal fails the decode.

An enum-indexed array surfaces as a Map, but on the wire it is still a dense array of one element per variant. Encoding a map that is missing a variant therefore fails rather than padding the gap with a zero.

A file with no top-level fields only declares types, and gets no struct of its own. All the .pdef files of a module generate into one package, so a type declared in one is usable from the others.

Checks

pdefGenerate checks every .pdef in the module together, and fails with each problem as file:line:column: message rather than generating code that will not compile - or, worse, code that compiles and reads the wrong bytes. It rejects:

• a field declared twice, at the top level or in one struct, which would otherwise drop a field from the layout without a word;
• a type declared twice, in one file or across files, including one that clashes with the struct named after a file, and two files with the same name;
• a reference to a type that is not declared anywhere, or an array indexed by something that is not an enum;
• an empty struct, an empty enum, an enum with a case repeated, and an enum of more than 256 cases;
• a negative size, or one too large for an Int, which are parse errors.

Differences from sbt-pdef-gen

Mostly a straight port. The behavioural changes are:

• Field order is preserved. The original kept top-level fields and type definitions in a HashMap, so the generated struct's field order — and hence the binary layout the codec implements — depended on hash iteration order. Both maps are now LinkedHashMap.
• string{n} is n bytes, not n bits. PdefStringCodec took its size in bits while the printer passed the declared number through unchanged, so string{16} produced a 2-byte field. pdefString now takes bytes.
• bool emits pdefBool. The original emitted scodec's bool(8), which writes true as all-ones (0xff) while the format stores 0x01. Decoding was unaffected, but re-encoding a record rewrote every true byte in it. pdefBool reads any non-zero byte as true and writes 0x01 back.
• float emits floatL. The original emitted scodec's big-endian float while every other scalar in the layout is little-endian.
• pdefEnumMap can encode. Its encode side folded updated over Vector.empty, which throws on the first element. It now writes the variants out in ordinal order.
• Definitions are checked before generating. The original declared UndefinedType but never raised it, and a repeated top-level field silently replaced the first.
• Keywords are whole words. intStats s used to read as an int named Stats.
• No enum macro. pdefEnum / pdefEnumMap take the enum's values, where the original summoned the cases with the allEnumSingletons macro. A byte that is no case's ordinal is now a failed decode rather than an IndexOutOfBoundsException.
• No BuildInfo. PdefModule.defaultPdefVersion carries the core coordinate that the sbt plugin got from sbt-buildinfo; keep it in sync with Versions.publish.

Naming moved with the port: package northstarcn.polaris2.pdefgen → nscn.polaris2.pdefgen, organization northstarcn.polaris2 → nscn.polaris2, and artifacts pdef-gen-runtime / sbt-pdef-gen → mill-pdef-core / mill-pdef-plugin_mill1.

Building

# unit tests; the `+` separates the two tasks, without it Mill reads
# `plugin.test` as a test-name selector for `core.test`
./mill core.test + plugin.test

# end-to-end; the example resolves the plugin from the local repository,
# so it has to be published first
./mill core.publishLocal + plugin.publishLocal
cd example && ./mill app.test

The plugin module's Scala version is pinned to what Mill 1.1.9 compiles build.mill with and is not free to move on its own — see the comment on plugin.scalaVersion. core is built with a newer Scala on purpose; consumers of the generated sources must be at or above it.

The example end-to-end check

example/app/pdef/Pdata231.pdef is the real Titanfall 2 persistence definition (19 enums, 29 structs, 165 top-level fields, ~13.8k leaf values once expanded), and example/app/test/resources/default.pdata is a real blob written by the game. Pdata231Tests decodes it with the generated codec, re-encodes it, and checks the result byte for byte.