|
PubChem PUG Help
1. Introduction.
The PubChem Power User
Gateway (PUG) provides access to PubChem services via a
programmatic interface. The basic design principle is straightforward. There
is a single CGI (pug.cgi, referred to hereafter as simply PUG) that is the
central gateway to multiple PubChem functions. PUG takes no URL arguments; all
communication with PUG is through XML. To perform any request, one formulates
input in XML and then HTTP POST it to PUG. The CGI interprets your incoming
request, initiates the appropriate action, then returns results (also) in XML
format. (This document assumes a basic familiarity with XML tags and data
structures. To learn more about XML, visit the URL:
http://en.wikipedia.org/wiki/XML)
PubChem services are queued. As
such, a submitted task will (usually) complete sometime after PUG responds to
the initial request. The initial PUG response contains the request ID of your
task. This request ID must be used for further communication with PUG
concerning your submitted task. When PUG is interrogated about an outstanding
request using the request ID, PUG will return either the results of your task,
if completed, or the status of your task.
Each PubChem service enabled for
use with PUG is documented separately. This service by service documentation
will detail the input, output, and options. All XML used by PUG is specified in
the data type definition (DTD), which may be found at:
http://pubchem.ncbi.nlm.nih.gov/pug/pug.dtd
or in the equivalent XML Schema
definition at:
http://pubchem.ncbi.nlm.nih.gov/pug/pug.xsd
We strongly recommend using an XML
parser/generator tool to read and write the XML data, rather than composing XML
manually.
PubChem PUG enabled services
have the ability to save and open valid PUG requests designed for
that service. You can use this feature to learn how to compose valid PUG XML
requests and to verify that your PUG XML request does what is intended. Examples of such services
provided in this document.
Additional documentation on
PubChem and its services may be found at
http://pubchem.ncbi.nlm.nih.gov and via help links throughout PubChem's web
site. If you cannnot find what you need there, further requests for information
or help may be sent to the highly knowledgeable and responsive NCBI help desk at
info@ncbi.nlm.nih.gov.
2. Interacting with PUG.
All communication to PUG is via
XML sent to the CGI at the URL:
http://pubchem.ncbi.nlm.nih.gov/pug/pug.cgi
The primary data container used in
all transactions is <PCT-Data>, the top-level container for any PUG input
or output. ("PCT" stands for PubChem Tools, a data specification that is shared
by both PUG and internal PubChem applications. See the introduction in this
document for more information.) This <PCT-Data> object may contain
either a <PCT-InputData> or a <PCT-OutputData> object. Users of
PUG will always send <PCT-Data> containing <PCT-InputData>, and
always receive <PCT-Data> containing <PCT-OutputData>.
After a new task is submitted to
PUG, your request is queued, rather than executing immediately. As such, PUG
will return an XML message containing a request ID to be used for further
actions on your request. In the example PUG XML reply below, the message says
that the request was successfully submitted and that the request ID is
"402936103567975582". It will then be up to you to (periodically) poll PUG,
using the request ID, until your task is complete. When your task is completed,
PUG will return the result; otherwise it will simply return a status message.
See examples in other sections of this document on how to properly poll PUG.
Example of a PUG reply to a newly submitted request:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_waiting>
<PCT-Waiting>
<PCT-Waiting_reqid>402936103567975582</PCT-Waiting_reqid>
</PCT-Waiting>
</PCT-OutputData_output_waiting>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
The <PCT-InputData> object
is a choice between request types. Tasks specific to various PubChem services
are contained by <PCT-InputData> and are described in different sections
of this document. Primary to the use of PUG is the <PCT-InputData> input
type used to perform request management, <PCT-Request>. Request
management enables you to enquire about the status of or to cancel a previous
PUG request. For example, to cancel a PUG request with request ID
"402936103567975582", the PUG XML input message will look like this:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_request>
<PCT-Request>
<PCT-Request_reqid>402936103567975582</PCT-Request_reqid>
<PCT-Request_type value="cancel"/>
</PCT-Request>
</PCT-InputData_request>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
The <PCT-OutputData>
object
contained in the output from PUG will always include a status message in a
<PCT-Status-Message>, which consists of an enumerated status in
<PCT-Status> and an optional message string. When a new task is queued by
PUG, the <PCT-OutputData> returned to you will (likely) contain a
<PCT-Waiting> which contains your request ID. If the request finishes
quickly, the initially returned <PCT-OutputData> object will actually
contain the appropriate result of your task specific to the requested service.
Similarly, when polling PUG using your request ID, the <PCT-OutputData>
object will contain either your task result or a status message.
3. PUG Tasks.
PubChem services currently enabled
for use by PUG include PubChem Download (http://pubchem.ncbi.nlm.nih.gov/pc_fetch/pc_fetch.cgi),
PubChem Compound Structure Search (http://pubchem.ncbi.nlm.nih.gov/search/search.cgi),
PubChem BioAssay Data Download (http://pubchem.ncbi.nlm.nih.gov/assay/assay.cgi), and PubChem Structure Standardization (http://pubchem.ncbi.nlm.nih.gov/standardize/standardize.cgi).
Each PUG service has its own expected input and provided output. The sections
below detail how to use each service with PUG.
3.1. PubChem
Substance/Compound Download Tasks.
This service allows you to
download sets of PubChem records - substances or compounds - using PUG's
<PCT-Download> sub-object. You will need to specify which records to
download, using a <PCT-QueryUids> object, the desired output format
(ASN.1, XML, or SDF), and, optionally, the desired compression method (gzip or
bzip2). The options available through PUG are equivalent to those for the
interactive PubChem Download service.
The <PCT-QueryUids> object
enables you to specify an explicit list of record IDs, or to provide an existing
Entrez history key (see eUtils documentation:
http://eutils.ncbi.nlm.nih.gov/entrez/query/static/eutils_help.html) from
either the PubChem Compound ("pccompound") or the PubChem Substance
("pcsubstance") Entrez databases. Currently there is an upper limit of 250,000
structures per download request; if you find this limit too restrictive for your
purposes, please consider using the PubChem FTP site which contains all
available PubChem contents:
ftp://ftp.ncbi.nlm.nih.gov/pubchem/
When your download request is
successfully completed, the returned <PCT-OutputData> object will hold a
<PCT-Download-URL> containing the URL you may use to download your
results. Again, please note that the result of a download task is an URL, not
the record data itself (which may be quite large). To obtain the data
requested, you must use the provided URL.
Example:
You want to download CID 1 and CID 99 – being uids 1 and 99 in the "pccompound"
Entrez database – in SDF format with gzip compression.
The typical flow of information is
as follows. First, the initial input XML is sent to PUG via HTTP POST. Note
the input data container with the download request and uid and format options:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_download>
<PCT-Download>
<PCT-Download_uids>
<PCT-QueryUids>
<PCT-QueryUids_ids>
<PCT-ID-List>
<PCT-ID-List_db>pccompound</PCT-ID-List_db>
<PCT-ID-List_uids>
<PCT-ID-List_uids_E>1</PCT-ID-List_uids_E>
<PCT-ID-List_uids_E>99</PCT-ID-List_uids_E>
</PCT-ID-List_uids>
</PCT-ID-List>
</PCT-QueryUids_ids>
</PCT-QueryUids>
</PCT-Download_uids>
<PCT-Download_format value="sdf"/>
<PCT-Download_compression value="gzip"/>
</PCT-Download>
</PCT-InputData_download>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
If the request is small and
finishes very quickly, you may get a final URL right away (see further below).
But usually PUG will respond initially with a waiting message and a request ID (<PCT-Waiting_reqid>)
such as:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_waiting>
<PCT-Waiting>
<PCT-Waiting_reqid>402936103567975582</PCT-Waiting_reqid>
</PCT-Waiting>
</PCT-OutputData_output_waiting>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
You would then parse out this request id, being "402936103567975582",
in this case, and use this id to "poll" PUG on the status of the request,
composing an XML message like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_request>
<PCT-Request>
<PCT-Request_reqid>402936103567975582</PCT-Request_reqid>
<PCT-Request_type value="status"/>
</PCT-Request>
</PCT-InputData_request>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
Note that here the request type
"status" is used; there is also the request type "cancel" that you may use to
cancel a running job.
If the request is still running,
you well get back another waiting message as above, and then you would poll again
after some reasonable interval. If the request is finished, you will get a final
result message like:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_download-url>
<PCT-Download-URL>
<PCT-Download-URL_url>
ftp://ftp-private.ncbi.nlm.nih.gov/pubchem/.fetch/1064385222466625960.sdf.gz
</PCT-Download-URL_url>
</PCT-Download-URL>
</PCT-OutputData_output_download-url>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
You would parse out the URL from
the <PCT-Download-URL_url> tag, and then use a tool of your choice to
connect to that URL to retrieve the actual requested data.
3.2. Query Tasks.
The <PCT-Query> object may
be used to perform queries against PubChem data that are not possible using
Entrez. The <PCT-Query> object consists of a series of queries and a
database to query against. One must be careful when formulating queries to
select compatible database and query types, as outlined in the documentation of
each query task type. For example, you will not want to perform a chemical
similarity search on a list of bioassay identifiers (AIDs), since chemical
searches may only be performed on compound identifiers (CIDs).
The <PCT-Query> object can
perform multiple queries in a single request. The semantic of multiple queries
in a single task is to "AND" the result between queries, which is to say that
the resulting list of identifiers will satisfy all queries requested. Sometimes
it is best to perform multiple search tasks individually rather than in a single
task, unless otherwise noted in the task documentation.
3.2.1. Chemical Structure Query
Tasks.
To perform PubChem Compound
structure searches using PUG, you will need to make a request using a
<PCT-Query> object. Chemical structure search tasks use the query objects
<PCT-QueryCompoundCS> and <PCT-QueryCompoundEL>. You may submit a
structure search by mixing and matching more than one of these two query types
in a series. Furthermore, only the PubChem Compound ("pccompound") Entrez
database may be specified in <PCT-QueryUids>, when performing chemical
structure queries.
The <PCT-QueryCompoundCS>
and <PCT-QueryCompoundEL> objects can encode many different types of
chemical structure searches. To help you understand how to encode a structure
search, please consider using the PubChem Structure Search web site. It has the
ability to translate your structure search query into the XML necessary for use
with PUG, and can be very helpful to demonstrate how to encode complex queries.
The PubChem Structure Search system is located at the URL:
http://pubchem.ncbi.nlm.nih.gov/search/
Please note that the output result
of a chemical structure search is an Entrez history key (see eUtils
documentation). To obtain the list of compounds matching your query, you must
use eUtils; more information on eUtils is below. There is currently a limit of
two million compound identifiers returned by the structure search (through
either PUG or the interactive web site).
Example:
You wish to perform a chemical similarity search of CID 2244 at a Tanimoto
similarity value of 80% with at most 300 results returned.
The initial HTTP POST to PUG to
initiate the search would contain XML like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_css>
<PCT-QueryCompoundCS>
<PCT-QueryCompoundCS_query>
<PCT-QueryCompoundCS_query_data>2244</PCT-QueryCompoundCS_query_data>
</PCT-QueryCompoundCS_query>
<PCT-QueryCompoundCS_type>
<PCT-QueryCompoundCS_type_similar>
<PCT-CSSimilarity>
<PCT-CSSimilarity_threshold>80</PCT-CSSimilarity_threshold>
</PCT-CSSimilarity>
</PCT-QueryCompoundCS_type_similar>
</PCT-QueryCompoundCS_type>
<PCT-QueryCompoundCS_results>300</PCT-QueryCompoundCS_results>
</PCT-QueryCompoundCS>
</PCT-QueryType_css>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
If the request is processed and
started successfully, PUG would respond with a waiting message and request id,
for example:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_waiting>
<PCT-Waiting>
<PCT-Waiting_reqid>271473836860076709</PCT-Waiting_reqid>
<PCT-Waiting_message>Structure search job was submitted</PCT-Waiting_message>
</PCT-Waiting>
</PCT-OutputData_output_waiting>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
You would then use this request ID, "271473836860076709"
in this case, to "poll" PUG for the status of the request:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_request>
<PCT-Request>
<PCT-Request_reqid>271473836860076709</PCT-Request_reqid>
<PCT-Request_type value="status"/>
</PCT-Request>
</PCT-InputData_request>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
If the search is still running,
you would get another waiting message as above, and you would then need to poll
again after a reasonable interval. If the search task is completed, PUG would
give an Entrez history key for the resulting CID (compound identifier) list:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
<PCT-Status-Message_message>Your
search has already been completed.</PCT-Status-Message_message>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_entrez>
<PCT-Entrez>
<PCT-Entrez_db>pccompound</PCT-Entrez_db>
<PCT-Entrez_query-key>1</PCT-Entrez_query-key>
<PCT-Entrez_webenv>
0Hm9YDD1X4wor4nONvSWx9vkKmEqFXTiq84JO47pgxmSw_
cIuDBVcG46Yr@2B5C47D162BBD720_0008SID
</PCT-Entrez_webenv>
</PCT-Entrez>
</PCT-OutputData_output_entrez>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
More information on using Entrez
history (and eUtils) to retrieve hit lists is below.
If for some reason your initial
query cannot be properly interpreted, PUG would respond with an error message
with some indication of the problem encountered:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="data-error"/>
</PCT-Status-Message_status>
<PCT-Status-Message_message>Programatic
Error:Non-decodeable query specified.
Input a valid SMILE/SMARTS or a CID.</PCT-Status-Message_message>
</PCT-Status-Message>
</PCT-OutputData_status>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
If you wish to cancel a queued or
running request, you would send to PUG:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_request>
<PCT-Request>
<PCT-Request_reqid>271473836860076709</PCT-Request_reqid>
<PCT-Request_type value="cancel"/>
</PCT-Request>
</PCT-InputData_request>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
And when PUG cancels your task,
you would get back:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="running"/>
</PCT-Status-Message_status>
<PCT-Status-Message_message>Your search will be stopped, please wait...
</PCT-Status-Message_message>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_waiting>
<PCT-Waiting>
<PCT-Waiting_reqid>271473836860076709</PCT-Waiting_reqid>
<PCT-Waiting_message>Your search will be stopped, please wait...
</PCT-Waiting_message>
</PCT-Waiting>
</PCT-OutputData_output_waiting>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
3.2.2. PubChem BioAssay Data Query and Download.
To perform PubChem BioAssay Data
Download using PUG, you will need to generate an input XML file. This XML file
has information about: what to download [being data for either particular
PubChem BioAssay identifiers (AIDs), particular AIDs and PubChem Compound
identifiers (CIDs), or particular AIDs and PubChem Substance identifiers (SIDs)];
the download format (XML, ASN.1, or CSV); and the output dataset type (complete
or concise). Eight examples
demonstrating the use of the BioAssay download tool are shown below.
Example 1:
You wish to download the complete BioAssay Data Table for AIDs 523 and 820
in CSV format. The initial HTTP POST to PUG to initiate the download would
contain XML as shown below.
The bioassay data may contain multiple screening results for the same chemical structure (represented as CID) which is deposited under several substance depositions (represented as SID). The "substance view" allows one to download the individual results for each sample of the chemical structure. It is recommended to download screening data from a single assay with this option. The "compound view" allows grouping the test results based on chemical structures. This option is particularly useful for downloading screening data from multiple assays.
This example shows the data in the substance view. To show the data in the compound view, you can replace '<PCT-Assay-FocusOption_group-results-by value="substance">4</PCT-Assay-FocusOption_group-results-by>' with '<PCT-Assay-FocusOption_group-results-by value="compound">0</PCT-Assay-FocusOption_group-results-by>'.
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_bas>
<PCT-QueryAssayData>
<PCT-QueryAssayData_output value="csv">4</PCT-QueryAssayData_output>
<PCT-QueryAssayData_aids>
<PCT-QueryUids>
<PCT-QueryUids_ids>
<PCT-ID-List>
<PCT-ID-List_db>pcassay</PCT-ID-List_db>
<PCT-ID-List_uids>
<PCT-ID-List_uids_E>523</PCT-ID-List_uids_E>
<PCT-ID-List_uids_E>820</PCT-ID-List_uids_E>
</PCT-ID-List_uids>
</PCT-ID-List>
</PCT-QueryUids_ids>
</PCT-QueryUids>
</PCT-QueryAssayData_aids>
<PCT-QueryAssayData_dataset value="complete">0</PCT-QueryAssayData_dataset>
<PCT-QueryAssayData_focus>
<PCT-Assay-FocusOption>
<PCT-Assay-FocusOption_group-results-by value="substance">4</PCT-Assay-FocusOption_group-results-by>
</PCT-Assay-FocusOption>
</PCT-QueryAssayData_focus>
</PCT-QueryAssayData>
</PCT-QueryType_bas>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
Example 2: You wish to download the concise BioAssay Data Table for AIDs 523 and 820 but just for the CIDs 3243128 and 3240114 in PubChem XML format. The initial HTTP POST to PUG to initiate the download would contain XML like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_bas>
<PCT-QueryAssayData>
<PCT-QueryAssayData_output value="assay-xml">1</PCT-QueryAssayData_output>
<PCT-QueryAssayData_aids>
<PCT-QueryUids>
<PCT-QueryUids_ids>
<PCT-ID-List>
<PCT-ID-List_db>pcassay</PCT-ID-List_db>
<PCT-ID-List_uids>
<PCT-ID-List_uids_E>523</PCT-ID-List_uids_E>
<PCT-ID-List_uids_E>820</PCT-ID-List_uids_E>
</PCT-ID-List_uids>
</PCT-ID-List>
</PCT-QueryUids_ids>
</PCT-QueryUids>
</PCT-QueryAssayData_aids>
<PCT-QueryAssayData_scids>
<PCT-QueryUids>
<PCT-QueryUids_ids>
<PCT-ID-List>
<PCT-ID-List_db>pccompound</PCT-ID-List_db>
<PCT-ID-List_uids>
<PCT-ID-List_uids_E>3243128</PCT-ID-List_uids_E>
<PCT-ID-List_uids_E>3240114</PCT-ID-List_uids_E>
</PCT-ID-List_uids>
</PCT-ID-List>
</PCT-QueryUids_ids>
</PCT-QueryUids>
</PCT-QueryAssayData_scids>
<PCT-QueryAssayData_dataset value="concise">1</PCT-QueryAssayData_dataset>
</PCT-QueryAssayData>
</PCT-QueryType_bas>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
Example 3: You wish to download the concise BioAssay Data Table for AIDs 523 and 820 but only for the SIDs 16952359 and 16952361 in PubChem ASN.1 format. The initial HTTP POST to PUG to initiate the download would contain XML like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_bas>
<PCT-QueryAssayData>
<PCT-QueryAssayData_output value="assay-text-asn">2</PCT-QueryAssayData_output>
<PCT-QueryAssayData_aids>
<PCT-QueryUids>
<PCT-QueryUids_ids>
<PCT-ID-List>
<PCT-ID-List_db>pcassay</PCT-ID-List_db>
<PCT-ID-List_uids>
<PCT-ID-List_uids_E>523</PCT-ID-List_uids_E>
<PCT-ID-List_uids_E>820</PCT-ID-List_uids_E>
</PCT-ID-List_uids>
</PCT-ID-List>
</PCT-QueryUids_ids>
</PCT-QueryUids>
</PCT-QueryAssayData_aids>
<PCT-QueryAssayData_scids>
<PCT-QueryUids>
<PCT-QueryUids_ids>
<PCT-ID-List>
<PCT-ID-List_db>pcsubstance</PCT-ID-List_db>
<PCT-ID-List_uids>
<PCT-ID-List_uids_E>16952359</PCT-ID-List_uids_E>
<PCT-ID-List_uids_E>16952361</PCT-ID-List_uids_E>
</PCT-ID-List_uids>
</PCT-ID-List>
</PCT-QueryUids_ids>
</PCT-QueryUids>
</PCT-QueryAssayData_scids>
<PCT-QueryAssayData_dataset value="concise">1</PCT-QueryAssayData_dataset>
</PCT-QueryAssayData>
</PCT-QueryType_bas>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
Example 4: You wish to download the complete BioAssay Data Table for AID 523 but only for those substances with "IC-50" value (the readout provided through TID 2) between 1 and 10 uM in CSV format. The initial HTTP POST to PUG to initiate the download would contain XML like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_bas>
<PCT-QueryAssayData>
<PCT-QueryAssayData_output value="csv">4</PCT-QueryAssayData_output>
<PCT-QueryAssayData_aids>
<PCT-QueryUids>
<PCT-QueryUids_ids>
<PCT-ID-List>
<PCT-ID-List_db>pcassay</PCT-ID-List_db>
<PCT-ID-List_uids>
<PCT-ID-List_uids_E>523</PCT-ID-List_uids_E>
</PCT-ID-List_uids>
</PCT-ID-List>
</PCT-QueryUids_ids>
</PCT-QueryUids>
</PCT-QueryAssayData_aids>
<PCT-QueryAssayData_dataset value="complete">0</PCT-QueryAssayData_dataset>
<PCT-QueryAssayData_readouts>
<PCT-Assay-Readout>
<PCT-Assay-Readout_aid>523</PCT-Assay-Readout_aid>
<PCT-Assay-Readout_query>
<PCT-Assay-Readout-Query>
<PCT-Assay-Readout-Query_fquery>
<PCT-Assay-Readout-Float-Query>
<PCT-Assay-Readout-Float-Query_tid>2</PCT-Assay-Readout-Float-Query_tid>
<PCT-Assay-Readout-Float-Query_lower>1</PCT-Assay-Readout-Float-Query_lower>
<PCT-Assay-Readout-Float-Query_upper>10</PCT-Assay-Readout-Float-Query_upper>
</PCT-Assay-Readout-Float-Query>
</PCT-Assay-Readout-Query_fquery>
</PCT-Assay-Readout-Query>
</PCT-Assay-Readout_query>
</PCT-Assay-Readout>
</PCT-QueryAssayData_readouts>
</PCT-QueryAssayData>
</PCT-QueryType_bas>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
Example 5: You wish to determine the TID number of "IC-50" for AID 523. The initial HTTP POST to PUG to initiate the download would contain XML like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_asd>
<PCT-QueryAssayDescription>
<PCT-QueryAssayDescription_aid>523</PCT-QueryAssayDescription_aid>
<PCT-QueryAssayDescription_get-columns value="true"/>
</PCT-QueryAssayDescription>
</PCT-QueryType_asd>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
Example 6: You wish to download the complete BioAssay Data Table of AID 523 with "Activity Outcome" as "Active" and "Activity Score" value between 10 and 40 in CSV format. The initial HTTP POST to PUG to initiate the download would contain XML like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_bas>
<PCT-QueryAssayData>
<PCT-QueryAssayData_output value="csv">4</PCT-QueryAssayData_output>
<PCT-QueryAssayData_aids>
<PCT-QueryUids>
<PCT-QueryUids_ids>
<PCT-ID-List>
<PCT-ID-List_db>pcassay</PCT-ID-List_db>
<PCT-ID-List_uids>
<PCT-ID-List_uids_E>523</PCT-ID-List_uids_E>
</PCT-ID-List_uids>
</PCT-ID-List>
</PCT-QueryUids_ids>
</PCT-QueryUids>
</PCT-QueryAssayData_aids>
<PCT-QueryAssayData_dataset value="complete">0</PCT-QueryAssayData_dataset>
<PCT-QueryAssayData_readouts>
<PCT-Assay-Readout>
<PCT-Assay-Readout_aid>523</PCT-Assay-Readout_aid>
<PCT-Assay-Readout_query>
<PCT-Assay-Readout-Query>
<PCT-Assay-Readout-Query_outcome>
<PCT-Assay-Activity-Outcome-Query value="active">2</PCT-Assay-Activity-Outcome-Query>
</PCT-Assay-Readout-Query_outcome>
</PCT-Assay-Readout-Query>
<PCT-Assay-Readout-Query>
<PCT-Assay-Readout-Query_score>
<PCT-Assay-Activity-Score-Query>
<PCT-Assay-Activity-Score-Query_lower>10</PCT-Assay-Activity-Score-Query_lower>
<PCT-Assay-Activity-Score-Query_upper>40</PCT-Assay-Activity-Score-Query_upper>
</PCT-Assay-Activity-Score-Query>
</PCT-Assay-Readout-Query_score>
</PCT-Assay-Readout-Query>
</PCT-Assay-Readout_query>
</PCT-Assay-Readout>
</PCT-QueryAssayData_readouts>
</PCT-QueryAssayData>
</PCT-QueryType_bas>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
Example 7: You wish to download the complete BioAssay Data Table of AID 523 with "IC-50" value (the readout provided through TID 2) between 1 and 10 micromolar in CSV format. You want to obtain just two bioassay (TID) columns (TID 1: IC50 Qualifier and TID 2: IC50). The initial HTTP POST to PUG to initiate the download would contain XML like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_bas>
<PCT-QueryAssayData>
<PCT-QueryAssayData_output value="csv">4</PCT-QueryAssayData_output>
<PCT-QueryAssayData_aids>
<PCT-QueryUids>
<PCT-QueryUids_ids>
<PCT-ID-List>
<PCT-ID-List_db>pcassay</PCT-ID-List_db>
<PCT-ID-List_uids>
<PCT-ID-List_uids_E>523</PCT-ID-List_uids_E>
</PCT-ID-List_uids>
</PCT-ID-List>
</PCT-QueryUids_ids>
</PCT-QueryUids>
</PCT-QueryAssayData_aids>
<PCT-QueryAssayData_dataset value="complete">0</PCT-QueryAssayData_dataset>
<PCT-QueryAssayData_readouts>
<PCT-Assay-Readout>
<PCT-Assay-Readout_aid>523</PCT-Assay-Readout_aid>
<PCT-Assay-Readout_retrieve>
<PCT-Assay-Readout_retrieve_E>2</PCT-Assay-Readout_retrieve_E>
<PCT-Assay-Readout_retrieve_E>1</PCT-Assay-Readout_retrieve_E>
</PCT-Assay-Readout_retrieve>
<PCT-Assay-Readout_query>
<PCT-Assay-Readout-Query>
<PCT-Assay-Readout-Query_fquery>
<PCT-Assay-Readout-Float-Query>
<PCT-Assay-Readout-Float-Query_tid>2</PCT-Assay-Readout-Float-Query_tid>
<PCT-Assay-Readout-Float-Query_lower>1</PCT-Assay-Readout-Float-Query_lower>
<PCT-Assay-Readout-Float-Query_upper>10</PCT-Assay-Readout-Float-Query_upper>
</PCT-Assay-Readout-Float-Query>
</PCT-Assay-Readout-Query_fquery>
</PCT-Assay-Readout-Query>
</PCT-Assay-Readout_query>
</PCT-Assay-Readout>
</PCT-QueryAssayData_readouts>
</PCT-QueryAssayData>
</PCT-QueryType_bas>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
Example 8: You wish to download the BioActivity Summary Table for CIDs 3243128 and 3240114 in AIDs 523 and 820, where the numbers of active, inactive compounds in each assay are listed. The initial HTTP POST to PUG to initiate the download would contain XML like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_qas>
<PCT-QueryActivitySummary>
<PCT-QueryActivitySummary_output value="summary-table">0</PCT-QueryActivitySummary_output>
<PCT-QueryActivitySummary_type value="assay-central">0</PCT-QueryActivitySummary_type>
<PCT-QueryActivitySummary_aids>
<PCT-QueryUids>
<PCT-QueryUids_ids>
<PCT-ID-List>
<PCT-ID-List_db>pcassay</PCT-ID-List_db>
<PCT-ID-List_uids>
<PCT-ID-List_uids_E>523</PCT-ID-List_uids_E>
<PCT-ID-List_uids_E>820</PCT-ID-List_uids_E>
</PCT-ID-List_uids>
</PCT-ID-List>
</PCT-QueryUids_ids>
</PCT-QueryUids>
</PCT-QueryActivitySummary_aids>
<PCT-QueryActivitySummary_scids>
<PCT-QueryUids>
<PCT-QueryUids_ids>
<PCT-ID-List>
<PCT-ID-List_db>pccompound</PCT-ID-List_db>
<PCT-ID-List_uids>
<PCT-ID-List_uids_E>3243128</PCT-ID-List_uids_E>
<PCT-ID-List_uids_E>3240114</PCT-ID-List_uids_E>
</PCT-ID-List_uids>
</PCT-ID-List>
</PCT-QueryUids_ids>
</PCT-QueryUids>
</PCT-QueryActivitySummary_scids>
</PCT-QueryActivitySummary>
</PCT-QueryType_qas>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
If one of the above requests is processed and started successfully, PUG would respond with a waiting message and request id, for example:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="running"/>
</PCT-Status-Message_status>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_waiting>
<PCT-Waiting>
<PCT-Waiting_reqid>336289408724569820</PCT-Waiting_reqid>
</PCT-Waiting>
</PCT-OutputData_output_waiting>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
You would then use this request ID, "336289408724569820" in this case, to "poll" PUG for the status of the request:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_request>
<PCT-Request>
<PCT-Request_reqid>336289408724569820</PCT-Request_reqid>
<PCT-Request_type value="status"/>
</PCT-Request>
</PCT-InputData_request>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
If the download is still running, you would get another waiting message as above, and you would then need to poll again after a reasonable interval. If the download task is completed, PUG would give an XML file containing assay description (Example 5), or a ftp URL (the rest examples) where the requested data is available:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_download-url>
<PCT-Download-URL>
<PCT-Download-URL_url>
ftp://ftp-private.ncbi.nlm.nih.gov/pubchem/.fetch/336289408724569820.csv.gz
</PCT-Download-URL_url>
</PCT-Download-URL>
</PCT-OutputData_output_download-url>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
If for some reason your initial query cannot be properly interpreted, PUG would respond with an error message with some indication of the problem encountered:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="server-error"/>
</PCT-Status-Message_status>
<PCT-Status-Message_message>
Status: server-error. Contact info@ncbi.nlm.nih.gov for assistance.
Please include your request ID if possible.
</PCT-Status-Message_message>
</PCT-Status-Message>
</PCT-OutputData_status>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
If you wish to cancel a queued or running request, you would send to PUG:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_request>
<PCT-Request>
<PCT-Request_reqid>336289408724569820</PCT-Request_reqid>
<PCT-Request_type value="cancel"/>
</PCT-Request>
</PCT-InputData_request>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
And when PUG cancels your task, you would get back:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="running"/>
</PCT-Status-Message_status>
<PCT-Status-Message_message>Request cancelled</PCT-Status-Message_message>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_waiting>
<PCT-Waiting>
<PCT-Waiting_reqid>336289408724569820</PCT-Waiting_reqid>
</PCT-Waiting>
</PCT-OutputData_output_waiting>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
3.3. PubChem Standardization
Tasks.
The PubChem Standardization
service allows you to standardize the representation of a chemical structure
using a <PCT-Standardize> sub-object. PubChem uses a normalization
procedure on all PubChem substance records to remove variation due to different
representations of functional groups, tautomeric or resonance forms, etc., to
create the PubChem Compound database, which contains the unique chemical
structures in the PubChem Substance database. This procedure verifies and
validates that a chemical structure is reasonable (to a certain degree) through
examination of the atoms and their valence and involves a valence-bond
canonicalization processing for tautomer invariance. The input to structure
standardization is a chemical structure and the output is either a failure
message or a chemical structure. To use this service, you will need to specify
an input structure and its format. You also need to specify the output format
you desire. This service operates on only a single structure at a time.
Example:
You would like to standardize the representation of guanine input in SMILES
format and output in SDF format.
The typical flow of information is
as follows. First, the initial input XML is sent to PUG via HTTP POST. Note
the input data container with the download request and uid and format options:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_standardize>
<PCT-Standardize>
<PCT-Standardize_structure>
<PCT-Structure>
<PCT-Structure_structure>
<PCT-Structure_structure_string>C1=NC2=C(N1)C(=O)N=C(N2)N
</PCT-Structure_structure_string>
</PCT-Structure_structure>
<PCT-Structure_format>
<PCT-StructureFormat value="smiles"/>
</PCT-Structure_format>
</PCT-Structure>
</PCT-Standardize_structure>
<PCT-Standardize_oformat>
<PCT-StructureFormat value="smiles"/>
</PCT-Standardize_oformat>
</PCT-Standardize>
</PCT-InputData_standardize>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
If the request is small and finishes very quickly, you
may get a final URL right away (see further below). But usually PUG will respond
initially with a waiting message and a request ID (<PCT-Waiting_reqid>)
such as:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_waiting>
<PCT-Waiting>
<PCT-Waiting_reqid>402936103567975582</PCT-Waiting_reqid>
</PCT-Waiting>
</PCT-OutputData_output_waiting>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
You would then parse out this request id, being "402936103567975582",
in this case, and use this id to "poll" PUG on the status of the request,
composing an XML message like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_request>
<PCT-Request>
<PCT-Request_reqid>402936103567975582</PCT-Request_reqid>
<PCT-Request_type value="status"/>
</PCT-Request>
</PCT-InputData_request>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
Note that here the request type
"status" is used; there is also the request type "cancel" that you may use to
cancel a running job.
If the request is still running,
you well get back another waiting message as above, and then you would poll again
after some reasonable interval. If the request is finished, you will get a final
result message like:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_structure>
<PCT-Structure>
<PCT-Structure_structure>
<PCT-Structure_structure_string>C1=NC2=C(N1)C(=O)N=C(N2)N
</PCT-Structure_structure_string>
</PCT-Structure_structure>
<PCT-Structure_format>
<PCT-StructureFormat value="smiles"/>
</PCT-Structure_format>
</PCT-Structure>
</PCT-OutputData_output_structure>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
You would parse out the output from the <PCT-Download-URL_url >
tag to retrieve the standardized structure.
4. PUG, NCBI eUtils, and Entrez
History.
NCBI's Entrez integrates the
scientific literature, DNA and protein sequence databases, 3D protein structure
and protein domain data, population study datasets, expression data, assemblies
of complete genomes, taxonomic information, and PubChem Compound, Substance, and
BioAssay databases (among others) into a tightly interlinked system. It is a
retrieval system designed for searching its linked databases. Entrez history
provides a record of the searches performed during a search session. PubChem
communicates with Entrez history through Entrez Programming Utilities (eUtils)
to enhance data analysis.
NCBI's eUtils are used extensively
by PubChem services. Results from queries are often provided in the form of an
Entrez history, which represents a list of database specific identifiers within
the Entrez search system. These identifiers are, for example, your PubChem CIDs
(compound identifiers). This allows you, the user, to interact with other
Entrez databases and to perform hit list management tasks using eUtils, e.g.,
to logically combine the results of different queries using AND, OR, or NOT
operations. PubChem services typically accept an Entrez history as a means to
provide a subset of identifiers as input, so that your query operates only on a
subset of a PubChem database contents. Use of Entrez history can help you avoid
sending and receiving (potentially) very large lists of identifiers. To learn
more about eUtils, please visit the URL:
http://eutils.ncbi.nlm.nih.gov/entrez/query/static/eutils_help.html
Histories in Entrez are database
specific. Each time an Entrez search is executed, the search terms, the time the
search was executed, and the search results are numbered consecutively and saved
automatically in Entrez history for that database. The history can be recalled
at any time during a search session, but histories are lost after 8 hours of
inactivity. There is also a running limit of 100 searches (across all databases)
saved in any given session.
PUG is integrated with Entrez in
that it may use Entrez history keys (also know as "webenv" keys) as both input
and output, depending on the task. For example, structure search via PUG may
return an Entrez history, and the resulting hit list can be retrieved as a list
of CIDs using Entrez's eFetch utility. PUG can also take a history key as
input, if you wanted to download the records resulting from either a prior
structure search or a programmatic Entrez search via the eSearch utility. (See
the eUtils documentation for more details.)
Entrez histories are referred to
programmatically by the trio of a database name, a WebEnv string, and a query
key number. You can see this in the example structure search above. The part of
PUG's response that contains this information is the <PCT-Entrez> tag:
<PCT-Entrez>
<PCT-Entrez_db>pccompound</PCT-Entrez_db>
<PCT-Entrez_query-key>1</PCT-Entrez_query-key>
<PCT-Entrez_webenv>
0Hm9YDD1X4wor4nONvSWx9vkKmEqFXTiq84JO47pgxmSw_
cIuDBVcG46Yr@2B5C47D162BBD720_0008SID
</PCT-Entrez_webenv>
</PCT-Entrez>
This Entrez history information
may be used in a variety of ways. If you want to view these hits on a regular
web page, you can direct a browser to an URL as follows, which shows the results
in HTML in the usual Entrez docsum format:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Select+from+History&
WebEnvRq=1&db=pccompound&query_key=1&
WebEnv=0Hm9YDD1X4wor4nONvSWx9vkKmEqFXTiq84JO47pgxmSw_
cIuDBVcG46Yr@2B5C47D162BBD720_0008SID
On the other hand, if you are
writing an application and want to retrieve the hit list directly via HTTP, you
can use eFetch with the same information, which can return the list in XML (with
its own DTD/XSD that is not related to PUG's), for example:
http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?
retmode=xml&rettype=uilist&
WebEnvRq=1&db=pccompound&query_key=1&
WebEnv=0Hm9YDD1X4wor4nONvSWx9vkKmEqFXTiq84JO47pgxmSw_
cIuDBVcG46Yr@2B5C47D162BBD720_0008SID
Finally, if you want to download
the compounds from this search in, e.g., SDF format with gzip
compression, you would send PUG a request with the <PCT-Entrez>
information instead of an explicit CID list. From here, the download process
would continue as in the example above.
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_download>
<PCT-Download>
<PCT-Download_uids>
<PCT-QueryUids>
<PCT-QueryUids_entrez>
<PCT-Entrez>
<PCT-Entrez_db>pccompound</PCT-Entrez_db>
<PCT-Entrez_query-key>1</PCT-Entrez_query-key>
<PCT-Entrez_webenv>
0Hm9YDD1X4wor4nONvSWx9vkKmEqFXTiq84JO47pgxmSw_
cIuDBVcG46Yr@2B5C47D162BBD720_0008SID
</PCT-Entrez_webenv>
</PCT-Entrez>
</PCT-QueryUids_entrez>
</PCT-QueryUids>
</PCT-Download_uids>
<PCT-Download_format value="sdf"/>
<PCT-Download_compression value="gzip"/>
</PCT-Download>
</PCT-InputData_download>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
PUG and eUtils together make
possible a wide variety of powerful programmatic data analysis tools for PubChem
and other Entrez databases.
5. PUG and SOAP.
There is a SOAP wrapper for PUG. Documentation and
a WSDL for which can be found at:
http://pubchem.ncbi.nlm.nih.gov/pug_soap
The interface includes much of PUG's functionality,
but with simplified functions that are accessible from GUI workflow applications
and SOAP-aware programming languages.
6. FAQs.
Some simple workflows help to illustrate the use of PUG. All PUG messages must be sent via a HTTP POST to the URL:
http://pubchem.ncbi.nlm.nih.gov/pug/pug.cgi
Scenario 1. I would like to retrieve the SMILES (in gzip compressed format) for a list of PubChem Compound CIDs: 1, 2, and 3.
Compose your PUG message:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_download>
<PCT-Download>
<PCT-Download_uids>
<PCT-QueryUids>
<PCT-QueryUids_ids>
<PCT-ID-List>
<PCT-ID-List_db>pccompound</PCT-ID-List_db>
<PCT-ID-List_uids>
<PCT-ID-List_uids_E>1</PCT-ID-List_uids_E>
<PCT-ID-List_uids_E>2</PCT-ID-List_uids_E>
<PCT-ID-List_uids_E>3</PCT-ID-List_uids_E>
</PCT-ID-List_uids>
</PCT-ID-List>
</PCT-QueryUids_ids>
</PCT-QueryUids>
</PCT-Download_uids>
<PCT-Download_format value="smiles"/>
<PCT-Download_compression value="gzip"/>
</PCT-Download>
</PCT-InputData_download>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
PUG will send you back a response, for example:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_waiting>
<PCT-Waiting>
<PCT-Waiting_reqid>402936103567975582</PCT-Waiting_reqid>
</PCT-Waiting>
</PCT-OutputData_output_waiting>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
This response contains a request ID, being "402936103567975582". You will use this request ID to query PUG on the status of your request by composing and sending another PUG message, as such:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_request>
<PCT-Request>
<PCT-Request_reqid>402936103567975582</PCT-Request_reqid>
<PCT-Request_type value="status"/>
</PCT-Request>
</PCT-InputData_request>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
If your request is still being processed, you will receive a response message as before. When your request completes, PUG will return an XML message like:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_download-url>
<PCT-Download-URL>
<PCT-Download-URL_url>
ftp://ftp-private.ncbi.nlm.nih.gov/pubchem/.fetch/656213441898678492.txt.gz
</PCT-Download-URL_url>
</PCT-Download-URL>
</PCT-OutputData_output_download-url>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
The PUG response gives you an URL where you may retrieve your results:
ftp://ftp-private.ncbi.nlm.nih.gov/pubchem/.fetch/656213441898678492.txt.gz
Scenario 2a. I have a SMILES of L-tyrosine and I would like to get back an SDF file containing the PubChem Compound record(s) exactly matching this structure (with the same stereo and isotopes).
The workflow for this scenario is very similar to Scenario 1, where you . Borrowing the workflow from the first scenario, the PUG message you HTTP POST would be:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_css>
<PCT-QueryCompoundCS>
<PCT-QueryCompoundCS_query>
<PCT-QueryCompoundCS_query_data>C1=CC(=CC=C1C[C@@H](C(=O)O)N)O
</PCT-QueryCompoundCS_query_data>
</PCT-QueryCompoundCS_query>
<PCT-QueryCompoundCS_type>
<PCT-QueryCompoundCS_type_identical>
<PCT-CSIdentity value="same-stereo-isotope">5</PCT-CSIdentity>
</PCT-QueryCompoundCS_type_identical>
</PCT-QueryCompoundCS_type>
<PCT-QueryCompoundCS_results>2000000</PCT-QueryCompoundCS_results>
</PCT-QueryCompoundCS>
</PCT-QueryType_css>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
The eventual result of the search will look something like:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
<PCT-Status-Message_message>Your search has completed successfully!
</PCT-Status-Message_message>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_entrez>
<PCT-Entrez>
<PCT-Entrez_db>pccompound</PCT-Entrez_db>
<PCT-Entrez_query-key>3</PCT-Entrez_query-key>
<PCT-Entrez_webenv>
0hJ--NzxiSKFxJzc4SnMb5PvxBP8HKJvZ-2s-XE19WBZSHG0xIO_k_xPrU@1FBE6DE17A397ED0_0011SID
</PCT-Entrez_webenv>
</PCT-Entrez>
</PCT-OutputData_output_entrez>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
When the query is complete, the result is an Entrez query key and webenv. The query key identifies the query result and the webenv provides your session identifier. You use this Entrez query key and webenv as your source of CIDs to compose another PUG message to download the gzipped compressed SDF file of the query hits:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_download>
<PCT-Download>
<PCT-Download_uids>
<PCT-QueryUids>
<PCT-QueryUids_entrez>
<PCT-Entrez>
<PCT-Entrez_db>pccompound</PCT-Entrez_db>
<PCT-Entrez_query-key>3</PCT-Entrez_query-key>
<PCT-Entrez_webenv>
0hJ--NzxiSKFxJzc4SnMb5PvxBP8HKJvZ-2s-XE19WBZSHG0xIO_k_xPrU@1FBE6DE17A397ED0_0011SID
</PCT-Entrez_webenv>
</PCT-Entrez>
</PCT-QueryUids_entrez>
</PCT-QueryUids>
</PCT-Download_uids>
<PCT-Download_format value="sdf"/>
<PCT-Download_compression value="gzip"/>
</PCT-Download>
</PCT-InputData_download>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
The final result, as in the first scenario, will contain an URL to the results containing CID 6057:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_download-url>
<PCT-Download-URL>
<PCT-Download-URL_url>
ftp://ftp-private.ncbi.nlm.nih.gov/pubchem/.fetch/693081357064045880.sdf.gz
</PCT-Download-URL_url>
</PCT-Download-URL>
</PCT-OutputData_output_download-url>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
Scenario 2b. I have a SMILES of L-tyrosine and I would like to get back an SDF file containing the PubChem Compound records containing the same isotopes (but stereo can be different).
This scenario is identical to Scenario 2a, except that the initial PUG message would look like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_css>
<PCT-QueryCompoundCS>
<PCT-QueryCompoundCS_query>
<PCT-QueryCompoundCS_query_data>
C1=CC(=CC=C1C[C@@H](C(=O)O)N)O</PCT-QueryCompoundCS_query_data>
</PCT-QueryCompoundCS_query>
<PCT-QueryCompoundCS_type>
<PCT-QueryCompoundCS_type_identical>
<PCT-CSIdentity value="same-isotope">4</PCT-CSIdentity>
</PCT-QueryCompoundCS_type_identical>
</PCT-QueryCompoundCS_type>
<PCT-QueryCompoundCS_results>2000000</PCT-QueryCompoundCS_results>
</PCT-QueryCompoundCS>
</PCT-QueryType_css>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
The final result would contain three records, being CIDs 6057, 1153, and 71098.
Scenario 2c. I have a SMILES of L-tyrosine and I would like to get back an SDF file containing the PubChem Compound records that have a similarity of 95%.
This scenario is identical to Scenario 2a, except that the initial PUG message would look like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_css>
<PCT-QueryCompoundCS>
<PCT-QueryCompoundCS_query>
<PCT-QueryCompoundCS_query_data>
C1=CC(=CC=C1C[C@@H](C(=O)O)N)O</PCT-QueryCompoundCS_query_data>
</PCT-QueryCompoundCS_query>
<PCT-QueryCompoundCS_type>
<PCT-QueryCompoundCS_type_similar>
<PCT-CSSimilarity>
<PCT-CSSimilarity_threshold>95</PCT-CSSimilarity_threshold>
</PCT-CSSimilarity>
</PCT-QueryCompoundCS_type_similar>
</PCT-QueryCompoundCS_type>
<PCT-QueryCompoundCS_results>2000000</PCT-QueryCompoundCS_results>
</PCT-QueryCompoundCS>
</PCT-QueryType_css>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
The final result would contain (currently) the SDF records for 191 CIDs.
Scenario 2d. How do I query using a molecular formula C2H7O and get back an SDF file containing the PubChem Compound records matching exactly?
This scenario is identical to Scenario 2a, except that the initial PUG message would look like:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_query>
<PCT-Query>
<PCT-Query_type>
<PCT-QueryType>
<PCT-QueryType_css>
<PCT-QueryCompoundCS>
<PCT-QueryCompoundCS_query>
<PCT-QueryCompoundCS_query_data>C2H7NO</PCT-QueryCompoundCS_query_data>
</PCT-QueryCompoundCS_query>
<PCT-QueryCompoundCS_type>
<PCT-QueryCompoundCS_type_formula>
<PCT-CSMolFormula></PCT-CSMolFormula>
</PCT-QueryCompoundCS_type_formula>
</PCT-QueryCompoundCS_type>
<PCT-QueryCompoundCS_results>2000000</PCT-QueryCompoundCS_results>
</PCT-QueryCompoundCS>
</PCT-QueryType_css>
</PCT-QueryType>
</PCT-Query_type>
</PCT-Query>
</PCT-InputData_query>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
The final result would contain (currently) the SDF records for 20 CIDs.
Scenario 3. How do I retrieve the SDF file of all PubChem Compound records within the mass range 100.00 to 100.01 atomic mass units?
Unlike the first two scenarios, you will query Entrez (not PUG) initially to generate the list of CIDs. To perform the Entrez query, using
"eSearch", use the URL:
http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pccompound&usehistory=y&retmax=0&term=100:100.01[exactmass]
Please note that the "retmax=0" argument in the URL above prevents the actual result list of PubChem Compound CIDs from being returned. The "usehistory=y" creates an Entrez history item, required for the next step in this scenario. If a CID list is desired, simply omit both, e.g.:
http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pccompound&term=100:100.01[exactmass]
The XML return message from eSearch, using the former URL above (rather than the latter), will look like:
<eSearchResult>
<Count>81</Count>
<RetMax>0</RetMax>
<RetStart>0</RetStart>
<QueryKey>26</QueryKey>
<WebEnv>
0BPLhFE_YfmLOCUMsO7FDRuhXLxgPqzfs-aB_O2nILEnCpSEb-AIRQzeQ0LTaQNNlpK8XkxiDcX71it@46C3203C79E0BE30_0000SID
</WebEnv>
<IdList>
</IdList>
<TranslationSet>
</TranslationSet>
<TranslationStack>
<TermSet>
<Term>0000100.000000[ExactMass]</Term>
<Field>ExactMass</Field>
<Count>-1</Count>
<Explode>Y</Explode>
</TermSet>
<TermSet>
<Term>0000100.010000[ExactMass]</Term>
<Field>ExactMass</Field>
<Count>-1</Count>
<Explode>Y</Explode>
</TermSet>
<OP>RANGE</OP>
</TranslationStack>
<QueryTranslation>0000100.000000[ExactMass] : 0000100.010000[ExactMass]</QueryTranslation>
</eSearchResult>
When the query is complete, the result is an Entrez query key and webenv.
The query key identifies the query result and the webenv provides your session identifier. You use this Entrez query key and webenv as your source of CIDs to compose another PUG message to download the gzipped compressed SDF file of the query hits:
<PCT-Data>
<PCT-Data_input>
<PCT-InputData>
<PCT-InputData_download>
<PCT-Download>
<PCT-Download_uids>
<PCT-QueryUids>
<PCT-QueryUids_entrez>
<PCT-Entrez>
<PCT-Entrez_db>pccompound</PCT-Entrez_db>
<PCT-Entrez_query-key>26</PCT-Entrez_query-key>
<PCT-Entrez_webenv>
0BPLhFE_YfmLOCUMsO7FDRuhXLxgPqzfs-aB_O2nILEnCpSEb-AIRQzeQ0LTaQNNlpK8XkxiDcX71it@46C3203C79E0BE30_0000SID
</PCT-Entrez_webenv>
</PCT-Entrez>
</PCT-QueryUids_entrez>
</PCT-QueryUids>
</PCT-Download_uids>
<PCT-Download_format value="sdf"/>
<PCT-Download_compression value="gzip"/>
</PCT-Download>
</PCT-InputData_download>
</PCT-InputData>
</PCT-Data_input>
</PCT-Data>
The final result, as in the first scenario, is an URL to the results:
<PCT-Data>
<PCT-Data_output>
<PCT-OutputData>
<PCT-OutputData_status>
<PCT-Status-Message>
<PCT-Status-Message_status>
<PCT-Status value="success"/>
</PCT-Status-Message_status>
</PCT-Status-Message>
</PCT-OutputData_status>
<PCT-OutputData_output>
<PCT-OutputData_output_download-url>
<PCT-Download-URL>
<PCT-Download-URL_url>
ftp://ftp-private.ncbi.nlm.nih.gov/pubchem/.fetch/816930703564580480.sdf.gz
</PCT-Download-URL_url>
</PCT-Download-URL>
</PCT-OutputData_output_download-url>
</PCT-OutputData_output>
</PCT-OutputData>
</PCT-Data_output>
</PCT-Data>
In this example, 81 hits are (currently) returned.
Scenario 4. How do I get back the PubMed abstracts for PubChem Compound CID 2244 (aspirin)?
Similar to Scenario 3, you will query Entrez (not PUG) to get a list of PubMed abstracts linked to a PubChem Compound. To perform the Entrez query, using
"eLink", for use the URL:
http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pccompound&id=2244&db=pubmed
Given that a list of abstracts may be long, it is a good idea to create an Entrez history item, e.g., using the URL:
http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pccompound&id=2244&db=pubmed&cmd=neighbor_history
The XML return message from eLink, using the latter URL above (rather than the former), will look like:
<eLinkResult>
<LinkSet>
<DbFrom>pccompound</DbFrom>
<IdList>
<Id>2244</Id>
</IdList>
<LinkSetDbHistory>
<DbTo>pubmed</DbTo>
<LinkName>pccompound_pubmed</LinkName>
<QueryKey>5</QueryKey>
</LinkSetDbHistory>
<LinkSetDbHistory>
<DbTo>pubmed</DbTo>
<LinkName>pccompound_pubmed_mesh</LinkName>
<QueryKey>6</QueryKey>
</LinkSetDbHistory>
<WebEnv>
0FFDpPFhSw2nzieR6fvaLMqXLnx9GKbcev9IJqI3EXp8pEzTEj38McL0EHmseTEpHXZkgacEGym2qtB@264F60B37ADDEBF0_0143SID
</WebEnv>
</LinkSet>
</eLinkResult>
In the message above, two Entrez history items were created, one corresponding to the depositor provided links (pccompound_pubmed) and those derived through linkage of MeSH ontology with depositor provided synonyms (pccompound_pubmed_mesh). To retrieve the abstracts, one may provide PubMed ids, one at a time or all at once, using
"eFetch".
To retrieve a single abstract, e.g., for PubMed id 12767473, one would formulate an URL like:
http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=12767473&retmode=xml&rettype=abstract
To retrieve abstracts for a list of PubMed ids contained in an Entrez history, e.g., for the eLink query above:
http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&
webenv=0FFDpPFhSw2nzieR6fvaLMqXLnx9GKbcev9IJqI3EXp8pEzTEj38McL0EHmseTEpHXZkgacEGym2qtB@264F60B37ADDEBF0_0143SID
&query_key=5&retmode=xml&rettype=abstract
The output of the above URLs is omitted for brevity.
Document Version History.
V1.3.0 – 2008June13 - Added new section on the PubChem
BioAssay Query/Download service. Minor cleanup of the previous documentation.
Corrected PUG SOAP URL.
V1.2.0 - 2008May20 - Added new section for FAQs,
providing usage scenarios. Updated the PUG SOAP documentation.
V1.1.0 –
2007Jan11 – Added new section on the PubChem Standardization service. Minor
cleanup of and additions to the previous documentation.
V1.0.0 –
2007May10 – Initial release.
| 
