Code2LoRA: Hypernetworks for Repository-Conditioned
and Commit-Streaming Adapters of Code Language Models

News

• Jul 2026 c2l-terminal demo submitted to EMNLP 2026 System Demonstrations.
• Jul 2026 Paper-exact benchmark datasets released on the Hugging Face Hub (artifacts).
• Jun 2026 Multi-task checkpoint: repository question answering added to the assertion-completion model via a forgetting-free warm start.
• May 2026 Code2LoRA-GRU checkpoint and interactive Space published.

Abstract

Code language models need repository-level context to resolve imports, APIs, and project conventions. Existing methods inject this knowledge as long inputs — retrieved through RAG or dependency analysis — or through per-repository fine-tuning: costly at repository scale and brittle to evolving codebases.

We introduce Code2LoRA, a hypernetwork framework that generates repository-specific LoRA adapters, effectively injecting repository knowledge with zero inference-time token overhead. Code2LoRA supports two usage scenarios:

We evaluate on RepoPeftBench, a benchmark of 512 Python repositories with 62K assertion-completion tasks spanning cross-repository (CR), in-repository (IR), and a held-out post-cutoff out-of-distribution (OOD) suite of 92 repositories created strictly after the training-scrape cutoff. Code2LoRA achieves 63.8% CR exact match — outperforming the pretrained Qwen2.5-Coder-1.5B backbone (45.7%), full fine-tuning (51.4%), retrieval-augmented generation (39.7%), and in-context learning (42.2%) — while adding zero inference tokens. On post-cutoff OOD repositories, Code2LoRA-GRU reaches 78.9% EM, a +33.3 absolute gain over the same pretrained backbone, evidencing that hypernetwork-generated adapters generalize to repositories that did not exist at training time.

c2l-terminal — the on-device demo system

c2l-terminal packages Code2LoRA-Evo as a self-contained, Claude-Code-style terminal assistant that runs entirely on a developer laptop. It walks your repository's commit history, generates a repository- and task-specific LoRA adapter in minutes on CPU (the base LLM is never loaded during generation), and serves completions from a quantized Qwen2.5-Coder-1.5B with that adapter injected.

$ pip install code2lora
$ cd ~/work/python-saml && c2lt --task qa --backend gguf

c2l-terminal — quantized Code2LoRA · base: Qwen2.5-Coder-1.5B (Q4_K_M)

c2l> /adapt
  mining commits ... 86 kept (new-assertion commits)
  embedding diffs + GRU walk ... done
  adapter ready for SAML-Toolkits/python-saml  ·  task: qa  ·  ~37 MB LoRA

c2l> Which new public method was introduced in OneLogin_Saml2_Auth
     for retrieving an attribute by FriendlyName?
`get_attribute_by_friendly_name(self, friendly_name)` was added. It returns
the first attribute whose friendly_name matches the provided value, or `None`
if no matching attribute is found.

c2l> /task assert_rhs
c2l> self.assertEqual(len(friendlyname_attributes),
0)

c2l> /adapt            # after a new commit: one embedding + one GRU step
  1 new commit folded into cached GRU state — adapter updated in seconds

Install

Published on PyPI as code2lora (Python ≥ 3.9, MIT license); installs the c2lt REPL and the c2l CLI:

# from PyPI
pip install code2lora

# optional: 4-/8-bit inference via bitsandbytes
pip install "code2lora[quant]"

# or from source
pip install git+https://github.com/lilianahotsko/c2l_terminal.git

Quick start (CPU laptop, GGUF backend)

# one-time: build llama.cpp and grab the quantized base model (~1 GB)
git clone https://github.com/ggerganov/llama.cpp ~/llama.cpp
cmake -B ~/llama.cpp/build -S ~/llama.cpp && cmake --build ~/llama.cpp/build -j
huggingface-cli download Qwen/Qwen2.5-Coder-1.5B-GGUF --include "*Q4_K_M*" \
    --local-dir ~/models/qwen-coder-gguf

export C2L_LLAMACPP=~/llama.cpp
export C2L_BASE_GGUF=~/models/qwen-coder-gguf/<the-q4-file>.gguf

# run inside any git repository
cd /path/to/your/repo
c2lt --device cpu --backend gguf
# c2l> /adapt          → generate the adapter (encoder + checkpoint auto-download, ~3 GB)
# c2l> /export-gguf    → convert the adapter for llama.cpp
# c2l> type a code prefix, or a question with /task qa

On a GPU (or with bitsandbytes ≥ 0.43 on CPU) you can skip llama.cpp entirely and use --backend 4bit. Full instructions, REPL command reference, and troubleshooting live in the GitHub README.

REPL commands

Method

Growing new tasks without forgetting — by construction

The shipped checkpoint started as a single-task assertion-completion model. We added free-form repository question answering by (1) generating 73,829 commit-grounded Q&A pairs over the training repositories with a batch LLM API for $28.50, and (2) growing the head with a zero-padded warm start: the new task-conditioning columns of the first trunk layer are initialized to zero, so the grown head is byte-identical to the source model for every task id at initialization — verified by a unit test on the generated LoRA tensors. Any forgetting is therefore attributable to fine-tuning drift alone, not to the surgery.

Results

Static track — cross-repository test (held-out repositories)

Per-repo LoRA requires a training run per repository and is undefined on unseen (CR/OOD) repositories — Code2LoRA matches its in-repo upper bound while generalizing across repositories with a single forward pass.

Evolution track — commit-streaming (GRU)

Out-of-distribution — repositories created after the training cutoff

92 GitHub repositories created strictly after the training-scrape cutoff — direct evidence that the hypernetwork captures repository structure rather than memorizing training-time repositories.

Repository question answering (multi-task checkpoint)

14,762 held-out commit-grounded Q&A pairs. The model reliably names the correct file and symbol; the strongest demo repositories retain 0.62–0.77 assertion EM after the QA fine-tune (e.g., jpsca/jinjax 0.77, fastapi/sqlmodel 0.75, SAML-Toolkits/python-saml 0.69). Full tables and ablations are in the papers.

Demo video

Artifacts

Everything is released under the MIT license on the 🤗 code2lora organization and GitHub. The benchmark datasets are paper-exact: they contain precisely the rows consumed by the reported experiments, post quality-filter.

Citation

If you use Code2LoRA, RepoPeftBench, or c2l-terminal, please cite:

@article{hotsko2026code2lora,
  title  = {Code2LoRA: Hypernetworks for Repository-Conditioned and
            Commit-Streaming Adapters of Code Language Models},
  author = {Hotsko, Liliana},
  year   = {2026},
  note   = {Under review}
}

@inproceedings{hotsko2026c2lterminal,
  title  = {c2l-terminal: An On-Device Coding Assistant that Generates
            Repository- and Task-Specific LoRA Adapters from Commit History},
  author = {Hotsko, Liliana},
  year   = {2026},
  note   = {EMNLP 2026 System Demonstrations, under review}
}