Giving LLMs a ’Molecular Cognition Module’: SciCore-Mol Tackles Heterogeneous Scientific Data

2026-05-08

Introduction

Does the pursuit of a "one-for-all" universal model mean a single Transformer must swallow every kind of data? Large language models (LLMs) excel at text, but when they encounter extremely complex heterogeneous scientific data such as molecular structures or protein folds, they easily fall into enormous computational waste and cognitive mismatch. A recent paper, SciCore-Mol, proposes a new approach: instead of forcibly textualizing molecules and feeding them to an LLM, it attaches several pluggable specialized modules to the model. Unlike tool learning, this approach fuses signals directly at the hidden-vector level, ensuring a unified, native understanding of molecular data inside the LLM. Experiments show that SciCore-Mol performs strongly across molecular understanding, generation, and reaction prediction, showing capabilities beyond GPT-5 at only 8B scale while preserving the LLM's general reasoning ability as much as possible. This work opens a new paradigm for inclusive AI4S.

➤ Paper

🔗 https://github.com/OpenBMB/SciCore-Mol/blob/main/SciCore_Mol_Technical_Report.pdf

➤ Source code and models

🔗 https://github.com/OpenBMB/SciCore-Mol

The Core Conflict: When a Text Model Meets the Geometric World

LLMs were originally designed to process discrete symbol sequences such as natural language, whereas molecular entities are fundamentally topological/spatial-geometric: interatomic connectivity, bond angles, and 3D conformations are extremely hard to losslessly compress into a string of characters.

Earlier work tried to adapt molecules to LLMs by linearizing molecular structures (SMILES, SELFIES), using a standard Transformer to understand high-dimensional geometry and other information. But this is mathematically very inefficient and easily causes two problems:

Information loss: key features such as spatial configuration and stereochemistry are erased;

Semantic noise: parentheses and numeric ring labels in linearized strings interfere with the LLM's normal reasoning.

Think of it this way: rather than having the "brain" understand a "picture" through exhaustive textual description, it is better to give it a sensory interface such as "eyes."

SciCore-Mol: Three Modules, Each with a Role

The paper proposes the SciCore-Mol architecture, using an LLM (further trained from Qwen3-8B) as the "central brain" with three pluggable cognition modules attached, each handling a different type of molecular signal:

Key design choices:

The LLM as the core: the LLM is no longer a passive decoder receiving information but a "central nervous system" with active scheduling authority. It dynamically decides which cognitive partition to activate based on the instruction.

Native perception instead of tool calls: these modules are not invoked through language (such as "call a tool API") but fused with the LLM at the hidden-vector level. The LLM is not asking for a result; it is sensing the scientific data object itself, achieving true semantic alignment.

Training Strategy: Three-Stage Progressive Optimization

Training all modules jointly with the LLM from the start easily causes gradient conflicts: different modules pull the optimization in different directions, making training highly unstable. The authors therefore designed a three-stage progressive strategy, letting each part first master its own specialty before gradually fusing them.

Stage 1: Molecular foundation pre-training

Each component is first pre-trained independently: the LLM base continues pre-training on chemistry-enhanced corpora, with a KL-divergence constraint to prevent forgetting general capabilities; GVP, DiT, and the reaction Transformer are pre-trained on large-scale molecular databases and open reaction databases, respectively. Next comes cross-modal alignment: for example, freeze the LLM and train only the GVP encoder and a lightweight MLP adapter. Through contrastive learning (NT-Xent loss), the molecular geometry encoding is aligned in latent space with the LLM's textual description of that molecule, teaching the LLM to "read" geometric signals.

Stage 2: Joint multi-task training

All modules are trained jointly on about 360,000 instruction-response pairs. Tasks cover name conversion, property prediction, molecule generation, synthesis prediction, retrosynthesis, molecule description generation, reaction product prediction, yield estimation, and stoichiometric reasoning, supplemented with general instruction-tuning data. In this stage, the LLM learns when to invoke which module.

Stage 3: Task-specific fine-tuning

For downstream applications, some modules can optionally be frozen. For example, for property prediction only, the diffusion and reaction modules can be frozen; for reaction tasks only, GVP and diffusion can be frozen. This design lets the model adapt flexibly to different scenarios without retraining the entire pipeline.

Innovations: Decoupled, Pluggable, and Without Sacrificing Generality

Heterogeneous data decoupling: text and geometric signals are processed on separate paths without interfering with each other, avoiding the semantic noise caused by "modality mismatch."

Flexible orchestration: modules can be enabled individually or combined as the task requires, without modifying the LLM's own parameters, maximally preserving its general capabilities.

Hidden-layer fusion: unlike conventional tool calls, this design lets the LLM perceive scientific data "natively," making reasoning smoother.

Progressive training strategy: pre-train each module, then jointly align and optimize, and finally optionally fine-tune — ensuring both module specialization and system-wide coordination.

Results: Strong Overall Performance and Outstanding Generalization

The paper evaluates SciCore-Mol on a range of tasks and common benchmarks including SMolInstruct, MMLU-Chem, ChemBench4K, ORD, and MoleculeNet. Against both domain models of similar scale and general closed-source models, SciCore-Mol shows very balanced performance; ablation studies also confirm the important value of the plug-in components. Notably, on DrugR, a newly released dataset for drug property optimization, the model demonstrates strong out-of-distribution generalization, fully validating its practical value.

Thanks to its modular decoupled design, SciCore-Mol solves the fine-tuning collapse common in scientific models. It can learn quickly from very small specialized datasets without disturbing the LLM's original reasoning foundation. This efficiency lets an 8B model deployable on-device match the level of ultra-large cloud models such as GPT-5 in molecular property prediction and reaction decision-making.

Implications: A Third Path for Domain Large Models

On the road to AGI, we need to think carefully about the role and meaning of domain large models. SciCore-Mol offers a third path distinct from "brute-force parameter scaling" and "simple linearization": enhancing the LLM's scientific cognition at the hidden-layer level through pluggable heterogeneous modules. This framework applies not only to molecules but can potentially extend to proteins, materials, physical fields, and other domains that handle heterogeneous scientific data. SciCore-Mol's success suggests that the endpoint of AI for Science may not be an ever-expanding monolithic black box, but an elegantly structured, self-evolving cognitive collaborative system.

Give the LLM senses and let it move from memorizing to understanding chemistry — perhaps that is the more elegant way for general models to enter the scientific domain.