Resolving a real dataset with Search¶

Search takes a column of identifiers and asks every installed offline database about each one. It returns one table with the structure the databases agree on, the identifiers they hold for it, where each came from, and how confident the answer is. It opens no network connection unless you ask it to.

This tutorial resolves a real dataset: the 1,144 compounds of the ESOL aqueous-solubility set (Delaney, J. Chem. Inf. Comput. Sci. 2004, 44, 1000–1005). The dataset gives a name and a SMILES for each compound, so every answer Search finds from the name can be checked against the structure the authors used.

The outputs below were produced on 2026-09-23 with all five databases installed. Installing the offline databases covers getting them. The reference for the arguments and columns is Resolving identifiers with Search.

In [1]:

import pandas as pd
from rdkit import Chem, RDLogger

from provesid import Search, datasets

RDLogger.DisableLog("rdApp.*")   # RDKit's per-molecule warnings, not ours
pd.set_option("display.width", 200)
pd.set_option("display.max_columns", 12)
pd.set_option("display.max_colwidth", 40)

import pandas as pd from rdkit import Chem, RDLogger from provesid import Search, datasets RDLogger.DisableLog("rdApp.*") # RDKit's per-molecule warnings, not ours pd.set_option("display.width", 200) pd.set_option("display.max_columns", 12) pd.set_option("display.max_colwidth", 40)

What is installed¶

Search uses the databases that are present and says which are missing. It does not download anything. datasets.status() shows what is on this machine:

In [2]:

datasets.status()[["dataset", "title", "present", "size", "release"]]

datasets.status()[["dataset", "title", "present", "size", "release"]]

Out[2]:

	dataset	title	present	size	release
0	pubchem	PubChem identifiers	True	2.2 GiB
1	comptox	EPA CompTox chemicals	True	1.1 GiB
2	chebi	ChEBI SDF	True	954.2 MiB
3	chembl	ChEMBL	True	30.1 GiB	36 (extract)
4	zeropm	ZeroPM inventory	True	438.7 MiB	0.0.4

One identifier¶

A CAS number, the most common case:

In [3]:

with Search("cas", show_progress=False) as s:
    one = s.search("50-78-2")

one[["query", "name", "InChIKey", "DTXSID", "source",
     "n_source_support", "confidence"]]

with Search("cas", show_progress=False) as s: one = s.search("50-78-2") one[["query", "name", "InChIKey", "DTXSID", "source", "n_source_support", "confidence"]]

Out[3]:

	query	name	InChIKey	DTXSID	source	n_source_support	confidence
0	50-78-2	acetylsalicylic acid	BSYNRYMUTXBXSQ-UHFFFAOYSA-N	DTXSID5020108	ChEBI	4	0.892

n_source_support is the number of independent databases that carry this structure, and confidence weighs it together with how the match was made. Every row also keeps a record of what each database returned:

In [4]:

one.loc[0, "source_details"]

one.loc[0, "source_details"]

Out[4]:

{'ChEBI': {'found': True,
  'fields': ['CASRN',
   'IUPAC_name',
   'InChI',
   'InChIKey',
   'SMILES',
   'Synonyms',
   'molecular_formula',
   'molecular_mass',
   'name']},
 'CompTox': {'found': True,
  'fields': ['CASRN',
   'DTXSID',
   'IUPAC_name',
   'InChIKey',
   'SMILES',
   'Synonyms',
   'molecular_formula',
   'molecular_mass',
   'name']},
 'PubChemID': {'found': True,
  'fields': ['CASRN',
   'IUPAC_name',
   'InChI',
   'InChIKey',
   'SMILES',
   'Synonyms',
   'molecular_formula',
   'molecular_mass',
   'name']},
 'ChEMBL': {'found': True,
  'fields': ['InChI',
   'InChIKey',
   'SMILES',
   'Synonyms',
   'molecular_mass',
   'name']}}

A whole table¶

The ESOL file has a name, two solubility columns and a SMILES:

In [5]:

esol = pd.read_csv("solubility_data_ESOL.csv")
print(len(esol), "compounds")
esol.head(3)

esol = pd.read_csv("solubility_data_ESOL.csv") print(len(esol), "compounds") esol.head(3)

1144 compounds

Out[5]:

	chemical_name	measured log(solubility:mol/L)	ESOL predicted log(solubility:mol/L)	SMILES
0	1,1,1,2-Tetrachloroethane	-2.18	-2.794	ClCC(Cl)(Cl)Cl
1	1,1,1-Trichloroethane	-2.00	-2.232	CC(Cl)(Cl)Cl
2	1,1,2,2-Tetrachloroethane	-1.74	-2.549	ClC(Cl)C(Cl)Cl

enrich adds the result columns to your own table, prefixed with provesid_, and resolves each distinct name once. Here it resolves by name only. The dataset's SMILES are kept aside for checking afterwards. On this machine it takes about three minutes.

In [6]:

%%time
with Search("name", show_progress=False) as s:
    found = s.enrich(esol, "chemical_name")

%%time with Search("name", show_progress=False) as s: found = s.enrich(esol, "chemical_name")

CPU times: user 2min 50s, sys: 2min 25s, total: 5min 15s
Wall time: 5min 26s

In [7]:

found.attrs["sources_available"], found.attrs["sources_unavailable"]

found.attrs["sources_available"], found.attrs["sources_unavailable"]

Out[7]:

(['chebi', 'comptox', 'pubchem', 'chembl'], [])

In [8]:

resolved = found["provesid_InChIKey"].notna()
print(f"{resolved.sum()} of {len(found)} names resolved offline")
found["provesid_n_source_support"].value_counts().sort_index()

resolved = found["provesid_InChIKey"].notna() print(f"{resolved.sum()} of {len(found)} names resolved offline") found["provesid_n_source_support"].value_counts().sort_index()

1038 of 1144 names resolved offline

Out[8]:

provesid_n_source_support
0    103
1     81
2    239
3    350
4    371
Name: count, dtype: int64

Most answers are carried by three or four databases. The 103 rows with zero support are names nothing matched. Three more matched a record with no single structure, and so have no InChIKey: tricresyl phosphate, for example, is a mixture of isomers. We come back to all of them below.

Checking the answers against the dataset¶

The InChIKey computed from the dataset's own SMILES can be compared with the InChIKey Search found from the name. Its first 14 characters (the skeleton) encode connectivity only, so two keys that share them differ in stereochemistry or charge, not in which molecule they describe.

In [9]:

def inchikey(smiles):
    mol = Chem.MolFromSmiles(smiles)
    return Chem.MolToInchiKey(mol) if mol else None

dataset_key = found["SMILES"].map(inchikey)
found_key = found["provesid_InChIKey"]

same = resolved & (found_key == dataset_key)
stereo_only = resolved & ~same & (found_key.str[:14] == dataset_key.str[:14])
different = resolved & (found_key.str[:14] != dataset_key.str[:14])

pd.Series({"identical": same.sum(),
           "same skeleton": stereo_only.sum(),
           "different molecule": different.sum()})

def inchikey(smiles): mol = Chem.MolFromSmiles(smiles) return Chem.MolToInchiKey(mol) if mol else None dataset_key = found["SMILES"].map(inchikey) found_key = found["provesid_InChIKey"] same = resolved & (found_key == dataset_key) stereo_only = resolved & ~same & (found_key.str[:14] == dataset_key.str[:14]) different = resolved & (found_key.str[:14] != dataset_key.str[:14]) pd.Series({"identical": same.sum(), "same skeleton": stereo_only.sum(), "different molecule": different.sum()})

Out[9]:

identical             902
same skeleton          99
different molecule     37
dtype: int64

In the "same skeleton" rows the keys differ only after the first 14 characters, which encode stereochemistry:

In [10]:

keys = pd.DataFrame({"name": found["chemical_name"], "from ESOL's SMILES": dataset_key,
                     "found": found_key, "source": found["provesid_source"]})
keys[stereo_only].set_index("name").loc[
    ["17a-Methyltestosterone", "1,8-Cineole", "Aldrin"]]

keys = pd.DataFrame({"name": found["chemical_name"], "from ESOL's SMILES": dataset_key, "found": found_key, "source": found["provesid_source"]}) keys[stereo_only].set_index("name").loc[ ["17a-Methyltestosterone", "1,8-Cineole", "Aldrin"]]

Out[10]:

	from ESOL's SMILES	found	source
name
17a-Methyltestosterone	GCKMFJBGXUYNAG-UHFFFAOYSA-N	GCKMFJBGXUYNAG-HLXURNFRSA-N	CompTox
1,8-Cineole	WEEGYLXZBRQIMU-UHFFFAOYSA-N	WEEGYLXZBRQIMU-WAAGHKOSSA-N	ChEBI
Aldrin	QBYJBZPUGVGKQQ-UHFFFAOYSA-N	QBYJBZPUGVGKQQ-SJJAEHHWSA-N	ChEBI

ESOL's SMILES carry no stereochemistry, and the databases define every centre.

The second block of a key also carries a flag: S if it was computed from a standard InChI, N if not. CompTox stores non-standard keys for about 11% of its substances and ZeroPM for about 5%. Such a key never equals the standard key of the same molecule, so Search replaces it with the standard key computed from the structure. Every key it returns is standard:

In [11]:

print("non-standard keys returned:", (found_key.dropna().str[23] != "S").sum())

print("non-standard keys returned:", (found_key.dropna().str[23] != "S").sum())

non-standard keys returned: 0

The rows where the molecules differ are the interesting ones:

In [12]:

cols = ["chemical_name", "SMILES", "provesid_name", "provesid_SMILES",
        "provesid_n_source_support", "provesid_confidence"]
found.loc[different, cols].sort_values("provesid_confidence")

cols = ["chemical_name", "SMILES", "provesid_name", "provesid_SMILES", "provesid_n_source_support", "provesid_confidence"] found.loc[different, cols].sort_values("provesid_confidence")

Out[12]:

	chemical_name	SMILES	provesid_name	provesid_SMILES	provesid_n_source_support	provesid_confidence
789	meconin	c1c(OC)c(OC)C2C(=O)OCC2c1	Meconin	COC1=C(OC)C2=C(COC2=O)C=C1	1	0.6314
379	alloxantin	C1(=O)NC(=O)NC(=O)C1(O)C2(O)C(=O)NC(...	Allantoin	NC(=O)NC1NC(=O)NC1=O	1	0.6460
1059	stadacaine	CCCCOc1ccc(C(=O)OCC)c(c1)N(CC)CC	Butoxycaine hydrochloride	Cl.CCCCOC1=CC=C(C=C1)C(=O)OCCN(CC)CC	1	0.6460
441	biquinoline	c2ccc1nc(ccc1c2)c4ccc3ccccc3n4	Biquinoline	C1=CC2=C(N=C1)C(=CC=C2)C1=CC=CC2=C1N...	1	0.6491
366	acetyl sulfisoxazole	CC(=O)N(S(=O)c1ccc(N)cc1)c2onc(C)c2C	SULFISOXAZOLE ACETYL	CC(=O)N(c1onc(C)c1C)S(=O)(=O)c1ccc(N...	1	0.6630
169	2,4,6-PCB	Clc1ccc(cc1)c2c(Cl)cccc2Cl	2,4,6-Trichlorobiphenyl	ClC1=CC(Cl)=C(C(Cl)=C1)C1=CC=CC=C1	1	0.6800
492	Chlorimuron-ethyl (ph 7)	CCOC(=O)c1ccccc1S(=O)(=O)NN(C=O)c2nc...	Chlorimuron-ethyl	CCOC(=O)C1=CC=CC=C1S(=O)(=O)NC(=O)NC...	1	0.6800
166	2,4,5-PCB	Clc1ccc(cc1)c2cc(Cl)ccc2Cl	2,4,5-Trichlorobiphenyl	ClC1=CC(Cl)=C(C=C1Cl)C1=CC=CC=C1	1	0.6800
145	2,3',4,4',5-PCB	Clc1ccc(c(Cl)c1)c2cc(Cl)c(Cl)c(Cl)c2Cl	2,3',4,4',5-Pentachlorobiphenyl	ClC1=CC(Cl)=C(C=C1Cl)C1=CC(Cl)=C(Cl)...	1	0.6800
772	L-arabinose	C1OC(O)C(O)C(O)C1O	aldehydo-L-arabinose	[H]C(=O)[C@H](O)[C@@H](O)[C@@H](O)CO	2	0.7190
289	3-Methyl-2-pentanol	CCC(C)CCO	3-methyl-2-pentanol	CCC(C)C(C)O	2	0.7334
288	3-Methyl-2-pentanol	CCC(C)CCO	3-methyl-2-pentanol	CCC(C)C(C)O	2	0.7334
611	dioctyl phthalate	CCCCCCCCOC(=O)c1ccccc1C(=O)OCCCCCCCC	bis(2-ethylhexyl) phthalate	CCCCC(CC)COC(=O)c1ccccc1C(=O)OCC(CC)...	2	0.7376
703	Fructose	OCC1OC(O)(CO)C(O)C1O	D-Fructose	OC[C@@H](O)[C@@H](O)[C@H](O)C(=O)CO	2	0.7505
599	Dimecron	CCN(CC)C(=O)C(=CCOP(=O)(OC)OC)Cl	Phosphamidon	CCN(CC)C(=O)C(Cl)=C(C)OP(=O)(OC)OC	2	0.7600
706	gentisin	c1c(O)C2C(=O)C3cc(O)ccC3OC2cc1(OC)	gentisin	COc1cc(O)c2c(=O)c3cc(O)ccc3oc2c1	2	0.7600
728	hydrazobenzene	N(Nc1ccccc1)c2ccccc2	phenylhydrazine	NNc1ccccc1	2	0.7600
998	Propyl propanoate	CCCCC(=O)OC	Propyl propionate	CCCOC(=O)CC	2	0.7600
958	phthalamide	c1cC2C(=O)NC(=O)C2cc1	phthalamide	NC(=O)c1ccccc1C(N)=O	2	0.7600
824	Methyl pentanoate	CCCC(=O)OCC	Methyl valerate	CCCCC(=O)OC	2	0.7600
501	chloropropylate	c1ccc(Cl)cc1C(c2ccc(Cl)cc2)(O)C(=O)O...	chloropropylate	CC(C)OC(=O)C(O)(c1ccc(Cl)cc1)c1ccc(C...	2	0.7600
836	Metolcarb	c1ccccc1(OC(=O)NC)	metolcarb	CNC(=O)Oc1cccc(C)c1	4	0.7714
870	Nitramine	CCN(CC)c1c(cc(c(N)c1N(=O)=O)C(F)(F)F...	N-methyl-N-picrylnitramine	CN(c1c([N+](=O)[O-])cc([N+](=O)[O-])...	4	0.7783
661	Ethyl pentanoate	CCCOC(=O)CCC	Ethyl pentanoate	CCCCC(=O)OCC	3	0.7821
934	Pentyl propanoate	CCCCC(=O)OCC	pentyl propanoate	CCCCCOC(=O)CC	3	0.7869
884	Norethisterone	CC34CCC1C(CCC2=CC(=O)CCC12O)C3CCC4(O...	norethisterone	[H][C@@]12CCC3=CC(=O)CC[C@]3([H])[C@...	4	0.7891
1131	triforine	ClC(Cl)(Cl)C(NC=O)N1C=CN(C=C1)C(NC=O...	triforine	[H]C(=O)NC(N1CCN(C(NC([H])=O)C(Cl)(C...	3	0.7905
996	Propyl butyrate	CCCC(=O)OC	propyl butyrate	CCCOC(=O)CCC	3	0.7905
612	Diosgenin	C1C(O)CCC2(C)CC3CCC4(C)C5(C)CC6OCC(C...	diosgenin	[H][C@@]12CC=C3C[C@@H](O)CC[C@]3(C)[...	3	0.8000
524	Coumaphos	CCOP(=S)(OCC)Oc2ccc1oc(=O)c(Cl)c(C)c1c2	coumaphos	CCOP(=S)(OCC)Oc1ccc2c(C)c(Cl)c(=O)oc2c1	4	0.8000
466	Butyl acetate	CCCCOC=O	butyl acetate	CCCCOC(C)=O	3	0.8000
295	3-Methylcholanthrene	c1cc(C)cc2c1c3cc4cccc5CCc(c45)c3cc2	3-methylcholanthrene	Cc1ccc2cc3c(ccc4ccccc43)c3c2c1CC3	4	0.8000
653	Ethyl butyrate	CCCCCOC(=O)CC	ethyl butyrate	CCCC(=O)OCC	3	0.8000
814	Methyl butyrate	CCCOC(=O)CC	Methyl butyrate	CCCC(=O)OC	3	0.8000
1022	Reposal	CCC1(C(=O)NC(=O)NC1=O)C2=CCC3CCC2C3	reposal	CCC1(C2=CC3CCC(C2)C3)C(=O)NC(=O)NC1=O	4	0.8000
1019	Quinonamid	ClC(Cl)CC(=O)NC2=C(Cl)C(=O)c1ccccc1C2=O	Quinonamid	O=C1C(Cl)=C(NC(=O)C(Cl)Cl)C(=O)c2ccc...	3	0.8000
1055	Simetryn	CSc1nc(nc(n1)N(C)C)N(C)C	simetryn	CCNc1nc(NCC)nc(SC)n1	4	0.8000

Reading them one by one, most are errors in the dataset, not in the lookup:

• Esters with another ester's SMILES. ESOL gives CCCCOC=O, butyl formate, for butyl acetate. In the block of simple esters, ethyl butyrate, methyl butyrate, propyl butyrate, ethyl and methyl pentanoate and pentyl and propyl propanoate each carry the SMILES of a different ester in the list. Two to four databases agree on each name's own structure.
• Wrong isomers. "3-Methyl-2-pentanol" is given as CCC(C)CCO, 3-methyl-1-pentanol. The SMILES for "2,4,5-PCB" and "2,4,6-PCB" put one chlorine on the other ring. Simetryn's SMILES is a different triazine.
• Malformed SMILES. Gentisin, meconin and phthalamide have SMILES that mix aromatic and aliphatic atoms in one ring, and they parse to a different molecule.

Some are the lookup's mistakes, and the columns point to them:

• Alloxantin came back as allantoin, from CompTox alone, at confidence 0.65: one source and a low score.
• Biquinoline names several isomers. ESOL means 2,2'-biquinoline, and the one database that answered gave 8,8'-biquinoline.
• Hydrazobenzene came back as phenylhydrazine, with two databases in support. That is a synonym error in the databases themselves, which no amount of corroboration catches.
• "Dioctyl phthalate" is used for both the n-octyl and the 2-ethylhexyl ester. The databases resolve it to the second, and ESOL meant the first.

So support and confidence tell you where to look first, not which side is right. The single-source answers at the top of the table include both the lookup's errors (alloxantin, biquinoline) and correct answers where the dataset is wrong (the PCBs, meconin). A structure two databases agree on can still be wrong when they share the same bad synonym.

Requiring corroboration¶

The "strict" preset accepts only structures that two or more databases agree on. Over the disputed names it drops exactly the single-source answers:

In [13]:

disputed = found.loc[different, "chemical_name"].tolist()

with Search("name", preset="strict", show_progress=False) as s:
    strict = s.search(disputed)

print(f"{strict['InChIKey'].notna().sum()} of {len(disputed)} disputed names "
      "still answered under 'strict'")
strict.loc[strict["InChIKey"].isna(), "query"].tolist()

disputed = found.loc[different, "chemical_name"].tolist() with Search("name", preset="strict", show_progress=False) as s: strict = s.search(disputed) print(f"{strict['InChIKey'].notna().sum()} of {len(disputed)} disputed names " "still answered under 'strict'") strict.loc[strict["InChIKey"].isna(), "query"].tolist()

28 of 37 disputed names still answered under 'strict'

Out[13]:

["2,3',4,4',5-PCB",
 '2,4,5-PCB',
 '2,4,6-PCB',
 'acetyl sulfisoxazole',
 'alloxantin',
 'biquinoline',
 'Chlorimuron-ethyl (ph 7)',
 'meconin',
 'stadacaine']

That removes the allantoin and biquinoline errors. It also removes the correct answers for the three PCBs, meconin, chlorimuron-ethyl and acetyl sulfisoxazole, whose ESOL SMILES are the ones at fault. Across the whole table it drops every correct answer that happens to be in only one database. Whether that trade is worth making depends on what a wrong structure would cost you.

The names nothing matched¶

In [14]:

missed = found.loc[~resolved, "chemical_name"]
missed.tolist()

missed = found.loc[~resolved, "chemical_name"] missed.tolist()

Out[14]:

['1,2,3,4-Tetrahydronapthalene',
 '1,5-Dimethlnapthalene',
 '1,7-phenantroline',
 '1-aminoacridine',
 '1-Bromonapthalene',
 '1-Chloronapthalene',
 '1-Iodonapthalene',
 '1-Napthylamine',
 '1-Nitronapthalene',
 "2,2',3,3',4,4',5,5',6,6'-PCB",
 "2,2',3,3',5,6-PCB",
 "2,2',3,4,4',5',6-PCB",
 "2,2',3,4,5,5',6-PCB",
 "2,2',3,4,5,5'-PCB",
 "2,2',3,4,5-PCB",
 "2,2',3,4,6-PCB",
 "2,2',3,5,5',6-PCB",
 "2,2',4,4',5,5'-PCB",
 "2,2,4,6,6'-PCB",
 "2,2',6,6'-PCB",
 "2,2'-PCB",
 "2,3,3',4,4',5-PCB",
 "2,3,3',4,4'6-PCB",
 "2,3',4,4'-PCB",
 "2',3,4-PCB",
 '2,4-PCB',
 '2,6-PCB',
 '2-Bromonapthalene',
 '2-Ethyl pyridine',
 '2-Ethyl-2-hexanal',
 '2-hydroxypteridine',
 '2-Methy-2-Butene',
 '2-Methyl-2-hexanol',
 '2-Methylnapthalene',
 '2-Methyltetrahydrofurane',
 '2-Napthol',
 '3,4-PCB',
 '3-Butanoyloxymethylphenytoin',
 '3-Ethanoyloxymethylphenytoin',
 '3-Heptanoyloxymethylphenytoin',
 '3-Hexanoyloxymethylphenyltoin',
 '3-Octanoyloxymethylphenytoin',
 '3-Pentanoyloxymethylphenytoin',
 '3-Propanoyloxymethylphenytoin',
 "4,4'-PCB",
 '5-(3-Methyl-2-butenyl)-5-ethylbarbital',
 '5-(3-Methyl-2-butenyl)-5-isoPrbarbital',
 '5,5-Diallylbarbital',
 '5,5-Diisopropylbarbital',
 '5-Allyl-5-ethylbarbital',
 '5-Allyl-5-isopropylbarbital',
 '5-Allyl-5-methylbarbital',
 '5-Allyl-5-phenylbarbital',
 '5-Ethyl-5-(3-methylbutyl)barbital',
 '5-Ethyl-5-phenylbarbital',
 '6-aminochrysene',
 '6-methoxypteridine',
 'Acenapthylene',
 'Amigdalin',
 'Antipyrene',
 'Benzyltrifluoride',
 'Chlorodibromethane',
 'cis 1,2-Dichloroethylene',
 'Cyclobutyl-5-spirobarbituric acid',
 'Cycloheptyl-5-spirobarbituric acid',
 'Cyclohexyl-5-spirobarbituric acid',
 'Cyclooctyl-5-spirobarbituric acid',
 'Cyclopentyl-5-spirobarbituric acid',
 'Cyclopropyl-5-spirobarbituric acid',
 'Di(2-ethylhexyl)-phthalate',
 'Digoxin (L1=41,8mg/mL, L2=68,2mg/mL, Z=40,1mg/mL)',
 'Diisopropylsulfide',
 'd-inositol',
 'Epitostanol',
 'Etoposide (148-167,25mg/ml)',
 'Fluorometuron',
 'Isonazid',
 'Lactose',
 'Malonic acid diethylester',
 'methyltestosterone acetate',
 'Metranidazole',
 'Napthacene',
 'Propylisopropylether',
 'Reverse Transcriptase inhibitor 1',
 'RTI 10',
 'RTI 11',
 'RTI 12',
 'RTI 13',
 'RTI 15',
 'RTI 16',
 'RTI 17',
 'RTI 19',
 'RTI 2',
 'RTI 20',
 'RTI 22',
 'RTI 23',
 'RTI 24',
 'RTI 3',
 'RTI 5',
 'RTI 6',
 'RTI 7',
 'RTI 9',
 'Sparsomycin (3,8mg/ml)',
 't-Pentylbenzene',
 'Trichlomethiazide',
 'Tricresyl phosphate']

Apart from the three mixtures, they fall into a few groups. There are misspellings ("napthalene", "Isonazid"), abbreviations no database indexes ("2,4-PCB", "RTI 10"), names with solubility annotations pasted in ("Etoposide (148-167,25mg/ml)"), and compounds that are simply not in any of the five databases (the spirobarbiturates and phenytoin prodrugs).

The "recall" preset turns on fuzzy name matching and ZeroPM, which is the one source that indexes common misspellings. It finds candidates to review by hand. It is much slower than an exact match, so here it runs on a few of the misses:

In [15]:

typos = ["1-Bromonapthalene", "2-Napthol", "Isonazid", "Metranidazole",
         "Trichlomethiazide", "Malonic acid diethylester", "Epitostanol",
         "RTI 11"]

with Search("name", preset="recall", n_hits=1, show_progress=False) as s:
    recall = s.search(typos)

recall[["query", "name", "match_method", "n_source_support", "confidence"]]

typos = ["1-Bromonapthalene", "2-Napthol", "Isonazid", "Metranidazole", "Trichlomethiazide", "Malonic acid diethylester", "Epitostanol", "RTI 11"] with Search("name", preset="recall", n_hits=1, show_progress=False) as s: recall = s.search(typos) recall[["query", "name", "match_method", "n_source_support", "confidence"]]

Out[15]:

	query	name	match_method	n_source_support	confidence
0	1-Bromonapthalene	1-Bromonaphthalene	fuzzy_name	2	0.7383
1	2-Napthol	2-Naphthol	fuzzy_name	2	0.7200
2	Isonazid	Isoniazid	fuzzy_name	1	0.6400
3	Metranidazole	Metronidazole	fuzzy_name	1	0.6277
4	Trichlomethiazide	Trichlormethiazide	fuzzy_name	1	0.6606
5	Malonic acid diethylester	oxalic acid diethylester	fuzzy_name	1	0.6106
6	Epitostanol	EICOSANOL	fuzzy_name	1	0.5440
7	RTI 11	Rti-112	fuzzy_name	1	0.6277

The first five are right. The last three are wrong: diethyl oxalate for diethyl malonate, eicosanol for epitostanol, and another RTI compound. A fuzzy match finds names that look alike, which is why this preset is for review and not for filling a table unattended.

Asking online¶

With online_fallback=True, a query that no offline source answered, and only such a query, is sent to PubChem and to the NCI/CADD resolver (CACTUS). This cell uses the network. Its output is from the run on the date above, and both services have bad days:

In [16]:

leftovers = ["2-Ethyl pyridine", "2,2'-PCB", "3,4-PCB", "5,5-Diallylbarbital"]

with Search("name", online_fallback=True, show_progress=False) as s:
    online = s.search(leftovers)

print("sent online:", online.attrs["online_fallbacks"],
      " answered:", online.attrs["online_resolved"])
online[["query", "name", "source", "n_source_support", "confidence"]]

leftovers = ["2-Ethyl pyridine", "2,2'-PCB", "3,4-PCB", "5,5-Diallylbarbital"] with Search("name", online_fallback=True, show_progress=False) as s: online = s.search(leftovers) print("sent online:", online.attrs["online_fallbacks"], " answered:", online.attrs["online_resolved"]) online[["query", "name", "source", "n_source_support", "confidence"]]

sent online: 4  answered: 3

Out[16]:

	query	name	source	n_source_support	confidence
0	2-Ethyl pyridine	2-Ethylpyridine	CACTUS	1	0.669
1	2,2'-PCB	1-chloro-2-(2-chlorophenyl)benzene	CACTUS	1	0.680
2	3,4-PCB	1,2-dichloro-4-phenylbenzene	CACTUS	1	0.680
3	5,5-Diallylbarbital	NaN	NaN	0	0.000

Rows that came from the network say so in source. They count as one vote in n_source_support, like a database.

How a result was made¶

Every frame records the settings that produced it and the sources that were used, so a saved table can say how it was made:

In [17]:

{k: found.attrs[k] for k in ("preset", "sources_available",
                             "online_fallbacks")}

{k: found.attrs[k] for k in ("preset", "sources_available", "online_fallbacks")}

Out[17]:

{'preset': 'balanced',
 'sources_available': ['chebi', 'comptox', 'pubchem', 'chembl'],
 'online_fallbacks': 0}

In [18]:

found.attrs["settings"]

found.attrs["settings"]

Out[18]:

{'fuzzy': False,
 'fuzzy_score_cutoff': 80.0,
 'fuzzy_scorer': 'ratio',
 'inchikey_skeleton': False,
 'similarity_threshold': 0.0,
 'sources': ('chebi', 'comptox', 'pubchem', 'chembl'),
 'top_k_per_source': 5,
 'cluster_by_skeleton': True,
 'consensus_compat_threshold': 0.35,
 'query_weight': 0.5,
 'n_hits': 1,
 'min_confidence': 0.0,
 'min_source_support': 0}

Where next¶

• Resolving identifiers with Search: every argument, every output column, and how confidence is computed.
• Using the local databases directly: asking one database at a time with PubChemID, CompToxID, ChebiSDF and CheMBL.
• The scripts in examples/search/ show single features in isolation: salt stripping, similarity search and the confidence arithmetic.