Taxonomy¶
Offline ChEBI classification with the optional Chebifier ensemble. See Chebifier taxonomy for installation.
provesid.taxonomy
¶
Chemical taxonomy classification backends for PROVESID.
This module provides high-level "give me chemical-class labels for these
structures" capabilities on top of PROVESID's identifier resolution. The first
backend implemented here is
ChebifierClassifier, a wrapper
around the offline, AI-based ChEB-AI chebifier ensemble that assigns ChEBI
ontology classes to molecules.
chebifier is a heavy, optional dependency (it pulls in PyTorch and, for the
graph models, part of the PyG stack). It is therefore not a core requirement
of PROVESID; install it with::
bash scripts/install_chebifier.sh
which wraps pip install 'chebifier[models]' plus the torch_scatter wheel
the graph models need from the PyG index.
See docs/guide/chebifier.md for the full installation story and known issues, and
plans/2026-07-02-chemical-taxonomy-classyfire-chebifier.md (§10) for the
design rationale.
Key design points (mirrors the PROVESID conventions):
- Optional + lazily imported. Importing
provesid.taxonomynever requires PyTorch.chebifieris imported only when aChebifierClassifieractually needs it, and a missing install raises a clearChebifierMissingErrorrather than a rawModuleNotFoundError. - Systemwide model storage. Model weights are redirected to the shared
per-user PROVESID dataset directory
(
provesid.utils.user_dataset_path, overridable withPROVESID_DATA_DIR) so a single copy is reused across all virtual environments on the machine, exactly like the other large PROVESID datasets. - InChIKey-keyed, resumable cache. Each structure is classified once and cached on disk (keyed by InChIKey + chebifier version + configuration), so a re-run over the same chemicals hits the cache and never reloads the model.
- Self-healing checkpoint compatibility. chebifier's graph checkpoints
require chebai-graph's property index vocabularies in a specific (older) state.
chebifier[models]==1.2.2pins a matchingchebai-graph, andensure_v244_indicesrestores the indices if a drifted version is installed over it.
Attributes¶
CHEBIFIER_PINNED_VERSION
module-attribute
¶
chebifier release this backend is validated against (see docs/guide/chebifier.md).
TAXONOMY_COLUMNS
module-attribute
¶
Columns of the tidy taxonomy table returned by classify (shared with the
planned ClassyFire backend, hence the ClassyFire-only level columns).
Classes¶
ChebifierError
¶
Bases: Exception
Base class for errors raised by the chebifier taxonomy backend.
Raised on its own for a bad ensemble configuration, such as an
exclude_models name the ensemble does not have.
Examples:
>>> ChebifierClassifier(exclude_models=["no_such_model"]).ensemble
Traceback (most recent call last):
...
provesid.taxonomy.ChebifierError: exclude_models names not in the ensemble configuration: ['no_such_model']. ...
Source code in src/provesid/taxonomy.py
103 104 105 106 107 108 109 110 111 112 113 114 115 | |
ChebifierMissingError
¶
Bases: ChebifierError
Raised when the optional chebifier dependency is not installed.
Raised when the ensemble is first built, not on import or construction;
check chebifier_available first
to avoid it.
Examples:
>>> issubclass(ChebifierMissingError, ChebifierError)
True
Source code in src/provesid/taxonomy.py
118 119 120 121 122 123 124 125 126 127 128 129 | |
ChebifierClassifier
¶
Classify molecules into ChEBI ontology classes with the chebifier ensemble.
Wraps the ChEB-AI chebifier ensemble behind PROVESID conventions:
systemwide model storage, an InChIKey-keyed resumable cache, and a tidy
pandas.DataFrame output. The (expensive) ensemble is constructed
lazily on first use and reused for the lifetime of the instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_dir
|
Optional[str]
|
Base directory for chebifier model storage. Defaults to
|
None
|
use_cache
|
bool
|
When |
True
|
resolve_names
|
bool
|
When |
False
|
model_configs
|
Optional[Union[str, Dict[str, Any]]]
|
Optional custom chebifier ensemble configuration (path or
dict) passed straight to |
None
|
patch_indices
|
bool
|
When |
True
|
with_scores
|
bool
|
When |
False
|
exclude_models
|
Optional[Sequence[str]]
|
Model names to drop from the ensemble configuration, e.g.
|
None
|
Raises:
| Type | Description |
|---|---|
ChebifierMissingError
|
If |
Examples:
>>> from provesid.taxonomy import ChebifierClassifier
>>> clf = ChebifierClassifier()
>>> df = clf.classify(["c1ccccc1", "CCO"])
>>> df["inchikey"].tolist()
['UHOVQNZJYSORNB-UHFFFAOYSA-N', 'LFQSCWFLJHTTHZ-UHFFFAOYSA-N']
>>> "30879" in df.loc[1, "chebi_ids"].split("|")
True
which takes the better part of a minute; later calls for the same structures come from the cache.
Source code in src/provesid/taxonomy.py
490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 | |
Attributes¶
chebifier_version
property
¶
Installed chebifier version (falls back to the pinned version).
Part of every cache key, so upgrading chebifier re-classifies.
Examples:
>>> ChebifierClassifier().chebifier_version
'1.2.2'
ensemble
property
¶
The lazily-constructed, reused BaseEnsemble instance.
Note
The ensemble is built with
data_dir as the working
directory, because chemlog_extra resolves its element-class
mapping files relative to the working directory (see
ensure_element_class_mappings).
The previous directory is always restored.
Raises:
| Type | Description |
|---|---|
ChebifierMissingError
|
If |
Examples:
>>> type(ChebifierClassifier().ensemble).__name__
'BaseEnsemble'
Methods:¶
predict_with_scores(smiles_list)
¶
Predict ChEBI classes and the score behind each decision.
BaseEnsemble.predict_smiles_list returns only the surviving class
ids; the smoothed net score it thresholds at 0 to get them is computed
and then discarded. This runs the same four steps
(gather_predictions -> consolidate_predictions ->
smoother -> > 0) and keeps the score, so callers get a per-label
confidence for free -- the models are run exactly once, no differently
and no more often than upstream does.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
smiles_list
|
Sequence[str]
|
Structures to classify. |
required |
Returns:
| Type | Description |
|---|---|
List[Optional[Dict[str, float]]]
|
One entry per input: a |
Note
Verified to reproduce predict_smiles_list's label sets exactly.
The scores are the smoothed ones, i.e. the values the ontology
consistency pass actually thresholds, so score > 0 holds for
every returned label. Not cached;
classify with
with_scores=True is.
Examples:
>>> scores = ChebifierClassifier().predict_with_scores(["CCO"])[0]
>>> scores["30879"], round(scores["2571"], 3)
(1.0, 0.866)
Source code in src/provesid/taxonomy.py
691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 | |
classify(smiles, inchikeys=None)
¶
Classify one or more structures into ChEBI ontology classes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
smiles
|
Union[str, Sequence[str]]
|
A SMILES string or a sequence of SMILES strings. |
required |
inchikeys
|
Optional[Sequence[Optional[str]]]
|
Optional pre-computed InChIKeys aligned with |
None
|
Returns:
| Type | Description |
|---|---|
DataFrame
|
A |
Raises:
| Type | Description |
|---|---|
ChebifierMissingError
|
If |
ValueError
|
If |
Examples:
>>> clf = ChebifierClassifier(with_scores=True)
>>> row = clf.classify("CCO").iloc[0]
>>> row["source"], row["chebi_ids"].split("|")[:3]
('chebifier', ['134179', '23367', '24431'])
>>> row["confidence"].split("|")[:3]
['1', '1', '1']
Source code in src/provesid/taxonomy.py
752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 | |
to_labels(df, level='chebi_ids')
staticmethod
¶
Collapse a taxonomy table to a {inchikey: label} mapping.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
A taxonomy table as returned by
|
required |
level
|
str
|
The column to use as the label (e.g. |
'chebi_ids'
|
Returns:
| Type | Description |
|---|---|
Dict[str, Optional[str]]
|
Mapping from InChIKey to the chosen label ( |
Raises:
| Type | Description |
|---|---|
KeyError
|
If |
Examples:
>>> table = pd.DataFrame({"inchikey": ["LFQSCWFLJHTTHZ-UHFFFAOYSA-N"],
... "chebi_ids": ["30879|33822"]})
>>> ChebifierClassifier.to_labels(table)
{'LFQSCWFLJHTTHZ-UHFFFAOYSA-N': '30879|33822'}
Source code in src/provesid/taxonomy.py
899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 | |
Functions:¶
chebifier_available()
¶
Return whether the optional chebifier package is importable.
This is a cheap check (it does not import PyTorch or load any model) suitable for feature-detection and for skipping tests when the extra is absent.
Returns:
| Type | Description |
|---|---|
bool
|
|
Examples:
>>> from provesid.taxonomy import chebifier_available
>>> isinstance(chebifier_available(), bool)
True
Source code in src/provesid/taxonomy.py
132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | |
missing_ensemble_modules()
¶
Modules required by the default chebifier ensemble that are not installed.
Use this to decide whether a full classification run can succeed.
chebifier_available only reports
whether chebifier itself imports, which is not enough: the default
ensemble also loads transformer, graph, rule-based and c3p models from
separate packages, and a partial install fails only once prediction is
attempted.
Returns:
| Type | Description |
|---|---|
List[str]
|
The missing module names, in the order they appear in the ensemble. Empty when the full default ensemble can be constructed. |
Examples:
>>> from provesid.taxonomy import missing_ensemble_modules
>>> missing_ensemble_modules()
['c3p']
Source code in src/provesid/taxonomy.py
166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 | |
default_ensemble_available()
¶
Whether every module the default chebifier ensemble needs is installed.
Returns:
| Type | Description |
|---|---|
bool
|
|
Examples:
>>> from provesid.taxonomy import default_ensemble_available
>>> default_ensemble_available() == (missing_ensemble_modules() == [])
True
Source code in src/provesid/taxonomy.py
192 193 194 195 196 197 198 199 200 201 202 203 204 205 | |
ensure_v244_indices()
¶
Ensure chebai-graph's property indices match chebifier's v244 checkpoints.
Reverts the BondType/AtomNumHs/NumAtomBonds one-hot vocabularies
inside the installed chebai_graph package to their pre-drift contents when
a newer (drifted) version is present. Without this, chebifier's graph (GNN)
models fail to load with a tensor-shape error. This is a safety net for a
hand-upgraded chebai_graph: the version pinned by chebifier[models]
(1.0.0) already matches, so every property reports "ok" on a clean
install. The operation is idempotent and a no-op when the indices already
match or when chebai_graph is not installed. See docs/guide/chebifier.md for
the full root-cause writeup.
Returns:
| Type | Description |
|---|---|
Dict[str, str]
|
Mapping of property name to a status string: |
Note
"patched" means a file inside the installed chebai_graph
package was rewritten.
Examples:
>>> ensure_v244_indices()
{'BondType': 'ok', 'AtomNumHs': 'ok', 'NumAtomBonds': 'ok'}
Source code in src/provesid/taxonomy.py
234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 | |
ensure_element_class_mappings(data_dir=None, chebi_version=244)
¶
Ensure chemlog_extra's element-to-ChEBI-class mapping files exist.
chemlog_extra's by-element classifiers read
data/chebi_v<version>/<Classifier>_element_class_mapping.csv relative to
the current working directory, and rebuild it from the ChEBI graph when the
file is absent. That rebuild crashes on the current graph — 288 of its 205k
nodes carry name: None, and the builder does
" molecular entity" in properties["name"] — so the chemlog models cannot
be constructed at all without these files, and the whole ensemble fails with
TypeError: argument of type 'NoneType' is not iterable.
This writes both files into the PROVESID chebifier data directory using
upstream's own derivation rules, skipping unnamed nodes.
ChebifierClassifier.ensemble
then builds the ensemble with that directory as the working directory, so
the files are found without writing anything into the caller's working
directory.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_dir
|
Optional[str]
|
Base directory for chebifier data. When |
None
|
chebi_version
|
int
|
ChEBI version the mappings are built for. Must match the version chemlog_extra asks for (its default, 244). |
244
|
Returns:
| Type | Description |
|---|---|
Dict[str, str]
|
Mapping of classifier name to a status string: |
Examples:
>>> from provesid.taxonomy import ensure_element_class_mappings
>>> ensure_element_class_mappings()
{'XMolecularEntityClassifier': 'ok', 'OrganoXCompoundClassifier': 'ok'}
Source code in src/provesid/taxonomy.py
293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 | |
chebi_class_names(data_dir=None)
¶
ChEBI id -> name for every term, read from chebifier's ontology snapshot.
chebifier already ships (and the ensemble already loads) a pickled networkx graph of the whole ChEBI hierarchy, whose nodes carry their names. Reading it is an offline alternative to resolving names one at a time through the online ChEBI client, which matters when labelling thousands of predicted classes at once.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_dir
|
Optional[str]
|
Base directory for chebifier storage; defaults to the shared
PROVESID dataset dir, exactly as
|
None
|
Returns:
| Type | Description |
|---|---|
Dict[str, str]
|
|
Raises:
| Type | Description |
|---|---|
ChebifierMissingError
|
If |
Examples:
>>> from provesid.taxonomy import chebi_class_names
>>> chebi_class_names()["33659"]
'organic aromatic compound'
Source code in src/provesid/taxonomy.py
426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 | |
classify_chebifier(smiles, inchikeys=None, *, data_dir=None, use_cache=True, resolve_names=False)
¶
Classify structures with the chebifier ensemble (convenience wrapper).
Thin functional wrapper over
ChebifierClassifier for one-off
calls. For repeated calls, construct a
ChebifierClassifier once and
reuse it so the model is loaded a single time.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
smiles
|
Union[str, Sequence[str]]
|
A SMILES string or sequence of SMILES strings. |
required |
inchikeys
|
Optional[Sequence[Optional[str]]]
|
Optional pre-computed InChIKeys aligned with |
None
|
data_dir
|
Optional[str]
|
Base directory for chebifier model storage. |
None
|
use_cache
|
bool
|
Whether to use the on-disk InChIKey cache. |
True
|
resolve_names
|
bool
|
Whether to resolve ChEBI ids to names. |
False
|
Returns:
| Type | Description |
|---|---|
DataFrame
|
A tidy taxonomy |
Raises:
| Type | Description |
|---|---|
ChebifierMissingError
|
If |
Examples:
>>> from provesid.taxonomy import classify_chebifier
>>> table = classify_chebifier("CCO")
>>> table.loc[0, "inchikey"]
'LFQSCWFLJHTTHZ-UHFFFAOYSA-N'
Source code in src/provesid/taxonomy.py
926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 | |