Ideally, we should also populate the map during decoding, in order to avoid re-encoding things that are already on disk.