Research

Optimal transport / efficient attention / ML systems

Research where mathematical structure meets hardware constraints.

I work on learning systems that have to survive both theoretical scrutiny and real machines: differentiable transport, long-context attention, sparse derivatives, and implementations designed around the memory hierarchy.

Featured paper · arXiv:2605.08123 · April 2026

Block-Wise Differentiable Sinkhorn Attention

Tail-refinement gradients with a gap-aware dustbin bridge: a memory-efficient route to differentiating balanced entropic optimal-transport attention at long context.

Block-wise transport attention

The active tile moves through banded support while the rest of the plan stays out of memory.

Your browser does not support Canvas. The paper uses a streamed block-wise transport schedule with one resident reference plan tile.

01 · Exact surrogate adjoint

One reference tile

For the reported two-step refinement path, the four staircase plan factors can be reconstructed from one resident reference tile and explicit row and column modifiers.

02 · Hardware-aware schedule

Stream the active support

Fixed-width block-wise execution yields O((T+R)LW) work, O(Ld) input storage, and O(L) additional HBM for fixed head dimension and band width.

03 · Structured gaps

Dustbin augmentation bridge

The implemented gap-aware path is formalized as the same balanced surrogate on an augmented support, allowing the adjoint schedule to lift without claiming a general unbalanced model.

Publications

Research record

This index is designed to grow as new papers are released, with each paper receiving its own explanation, artifacts, and evidence boundary.

2026

Preprint

Block-Wise Differentiable Sinkhorn Attention: Tail-Refinement Gradients with a Gap-Aware Dustbin Bridge

Dylan Forde

Earlier research threads

Problems that shaped the current work

Deep reinforcement learning

Intrinsic quantification of domain shift

Reactive exploration methods for measuring domain-shift magnitude across controlled reinforcement-learning environments.

Computational biology

Multi-omics and biological representation learning

Research across biomarker discovery, protein representations, regulatory networks, and learning from heterogeneous biological signals.

Medical imaging

Attention across volumetric MRI

Patch tokenization, self-attention, and inter-slice aggregation for volume-aware diagnostic classification.