Scaling an Autoregressive Transformer for Single-Cell Generation
Quick summary
arXiv:2608.02961v1 Announce Type: cross Abstract: We study a self-supervised generation task for single-cell gene expression vectors: given a set of vectors from a cell type, we aim to generate additional gene expression vectors of that cell type. For this task we characterize both the biological fidelity of the generated gene expression vectors and the scaling behavior of the pretraining loss. The model is a causal transformer paired with a learned quantized VAE tokenizer, trained with a cross-entropy loss. To evaluate the model, we condition it on held-out gene expression vectors of a cell t
Key takeaways
- arXiv:2608.02961v1 Announce Type: cross Abstract: We study a self-supervised generation task for single-cell gene expression vectors: given a set of vectors from a cell type, we aim to generate additional gene expression vectors of that cell type.
- For this task we characterize both the biological fidelity of the generated gene expression vectors and the scaling behavior of the pretraining loss.
- The model is a causal transformer paired with a learned quantized VAE tokenizer, trained with a cross-entropy loss.
Why it matters
“Scaling an Autoregressive Transformer for Single-Cell Generation” highlights the need for repeatable measurement rather than a single impressive demonstration. Independent validation across datasets and clearly stated limitations determine whether a result can guide product decisions.
Member comments