ISONE wishes to lay out this authentication method and relevant
application protocols as a specification so that customers can
develop their own programmatic clients (or obtain them from
third-party vendors). In addition, we will include whatever
implementation notes or troubleshooting tips we have collected from
our experience with these HTTPS clients. This document covers the
SMS applications (IBTs, Meter Readings, Daily Tags).
Please note that the SMS application protocols are currently based
entirely on UI behavior, and are subject to change as new UI
features are introduced. A separate protocol for programmatic
clients would require a new project to develop it, and this project
would have to be supported by the whole ISONE customer base.
2. Specification
2.1 Authentication method
The client must connect to ISONE's web server for SMD Applications
(currently at smd.iso-ne.com) via HTTPS using authenticated SSL.
The client must authenticate itself using a digital certificate
issued by ISONE. The certificate must be currently valid, and must
be authorized by ISONE to access the specific SMD applications
requested.
Certificate holders are authorized on an application-by-application basis, so, for example, an individual certificate holder may be allowed to access https://smd.iso-ne.com/sms_oper_contract/main (IBTs) but not https://smd.iso-ne.com/sms_oper_metering/main (Meter Readings). The authorizations also depend on the SMD environment. For instance, SMD Sandbox (currently at sandboxsmd.iso-ne.com) can be accessed using the same clients, but could potentially have different authorizations from smd.iso-ne.com. There are other differences between these environments as well (such as server ID certificates), so the developer must always be certain about which environment his client is authenticating itself to.
The SSL protocols accepted by the ISONE SMD web server are SSLv3 and TLSv1. SSLv2 is not accepted. Both the client and server certificates contain RSA private keys, and therefore RSA authentication is always used (as opposed to DSA, for instance). The accepted cipher suites for encryption include are (in order of strength):
| Cipher-Tag | Key Exchange | Encryption | Mac | Strength |
|---|---|---|---|---|
| EDH-RSA-DES-CBC3-SHA | DH | 3DES(168) | SHA1 | Strong |
| DES-CBC3-SHA | RSA | 3DES(168) | SHA1 | Strong |
| RC4-SHA | RSA | RC4(128) | SHA1 | Moderate |
| RC4-MD5 | RSA | RC4(128) | MD5 | Moderate |
| EDH-RSA-DES-CBC-SHA | DH | DES(56) | SHA1 | Moderate |
| DES-CBC-SHA | RSA | DES(56) | SHA1 | Moderate |
Request URL: "https://smd.iso-ne.com/sms_oper_contract/main"
Request method: POST
Request Cookie header: <SMS_STL_SESSIONID>
Request Content-Type header: "multipart/form-data; boundary=<boundary string>"
Request post data (with newlines):
--<boundary string> Content-Disposition: form-data; name="file"; filename="<name of contract upload XML file>" Content-Type: text/xml <body of contract upload XML file> --<boundary string>--Response code: 200
...
<table>
<tr><td><b>Files uploaded with Warnings:</b></td></tr>
<ul>
<tr><td><li><name of contract upload XML file></li></td></tr>
...
Warning Messages:
<textarea>
<---Contract ID XXXXX
Effective 1/1/2013, the final acceptance of <name of contract type> Bilateral Contracts is subject to the financial assurance review provisions in Section III.B.3(e) of the ISO New England Financial Assurance Policy.>
Response data for unsuccessful upload contains:
...
<table>
<tr><td><b>Files failed to upload:</b></td></tr>
<ul>
<tr><td><li><name of contract upload XML file></li></td></tr>
...
Error messages:
<textarea>
<Error messages for user>
Downloads XML containing contracts and schedules for contracts of a given contract type within a given date range. The contract types are based on values listed in the "Search For Existing Contract" screen of the IBT UI, and are currently:
servlet_action=SEARCH_CONTRACT &contractIDField= &contractTypeCheck=contractTypeCheck &contractTypeSelect=<requested contract type> &refIDField= &marketButton=DAMarket &marketCheck: marketCheck &startDateCheck=startDateCheck &startDateField=<requested date range start date> &locationField= &location_type= &endDateCheck=endDateCheck &endDateField=<requested date range end date> &statusButton=PENDING_REQUESTOR &party_name= &partyField= &schedPatternButton= On-Peak+5x16 &schedStatusButton=PENDING_REQUESTOR &scheduleField= &subaccountIdField= &asset_name= &assetField= &commitment_period_id_field= &resource_name= &resourceField=Response code: 200
... <a onClick="return(showContract(this, matching contract number));"> ...
servlet_action=DOWNLOAD_FILE &fileContent=contractsAndSchedules &fileFormat=xmlResponse code: 200
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Download_ContractsAndSchedules PUBLIC "-//ISO New England, Inc//DTD Contract and Schedules Download//EN"
"">
<Download_ContractsAndSchedules>
<Contract>
...
</Contract>
...
</Download_ContractsAndSchedules>
... <td class="lfr_title" align="center" width="450"><b>Forward Reserve Asset Assignment<b></td> ... Lead Participant:</td> <td width="300" ><b>name of lead participant</b> ...
Request URL: "https://smd.iso-ne.com/sms_oper_lfrassetassign/fileUpload"
Request method: GET
Request Cookie header: <SMS_STL_SESSIONID>
Response data contains:
... <div class="upload_frame"> <form name="UploadForm" ....><input type="hidden" name="_csrf" value="token value"> ...
Request URL: "https://smd.iso-ne.com/sms_oper_lfrassetassign/fileUpload/submit"
Request method: POST
Request Cookie header: <SMS_STL_SESSIONID>
Request Content-Type header: "multipart/form-data; boundary=<boundary string>"
Request post data (with new lines):
--<boundary string> Content-Disposition: form-data; name="_csrf" -<token value used from earlier GET /fileUpload response.> --<boundary string> Content-Disposition: form-data; name="browseFile"; filename="<name of lfr asset assignment upload XML file>" Content-Type: text/xml <body of lfr asset assignment upload XML file> --<boundary string> Content-Disposition: form-data; name="submitFile" Submit --<boundary string> Content-Disposition: form-data; name="action" FILEUPLOAD --<boundary string>--Response code: 200
...
<div class="upload_results_frame">
...
<td colspan=2 valign=top>
Successfully submitted.<br>
==============================<br>
<br>
Customer ID: <submitter's customer id><br>
Asset ID: <asset id><br>
Reserve Zone ID: <reserve zone id><br>
Product Type: <TMNSR|TMOR> <br>
Date: <assignment date>
<table border=0>
<table of submitted data.>
...
<repeated for additional assets>
Response data for unsuccessful upload contains:
<td valign=top>
<center> Submission Failed, All records rejected: <br>
==============================<br>
<br>
Customer ID: <submitter's customer id><br>
Asset ID: <asset id><br>
Reserve Zone ID: <reserve zone id><br>
Product Type: <TMNSR|TMOR> <br>
Date: <assignment date>
<table border=0>
<table of submitted data.>
</table>
<Reason(s) for submission failure>
Request URL: "https://smd.iso-ne.com/sms_oper_lfrassetassign/fileDownload?beginDate=yyyy-mm-dd&endDate=yyyy-mm-dd"
Request method: GET
Request Cookie header: <SMS_STL_SESSIONID>
Response code: 200
Content-Disposition: attachment;filename="asset_assignments.txt";
Response data contains:
download file contents
...
<td class="lfr_title" align="center" width="450"><b>Forward Reserve Bidding<b></td>
...
<input type="button" class="lfr_button" value="My Portfolio" width="50" onclick="javascript:setControllerMapping('myPortfolio')">
...
Participant: <b>name of participant</b>
...
Request URL: "https://smd.iso-ne.com/sms_oper_lfrbidding/fileUpload"
Request method: GET
Request Cookie header: <SMS_STL_SESSIONID>
Response data contains:
... <div class="upload_frame"> <form name="UploadForm" ....><input type="hidden" name="_csrf" value="token value"> ...
Request URL: "https://smd.iso-ne.com/sms_oper_lfrbidding/fileUpload/submit"
Request method: POST
Request Cookie header: <SMS_STL_SESSIONID>
Request Content-Type header: "multipart/form-data; boundary=<boundary string>"
Request post data (with new lines):
--<boundary string> Content-Disposition: form-data; name="_csrf" -<token value used from earlier GET /fileUpload response.> --<boundary string> Content-Disposition: form-data; name="browseFile"; filename="<name of lfr bidding upload XML file>" Content-Type: text/xml <body of lfr bidding upload XML file> --<boundary string> Content-Disposition: form-data; name="submitFile" Submit --<boundary string> Content-Disposition: form-data; name="action" FILEUPLOAD --<boundary string>--Response code: 200
...
<div class="upload_results_frame">
...
<td colspan=2 valign=top>
Successfully submitted.<br>
==============================<br>
<br>
Offer ID: <Unigue identifier for an Offer><br>
Customer ID: <submitter's customer id><br>
Auction Start Date: <Auction Begin date><br>
Auction End Date: <Auction End date><br>
Reserve Zone ID: <reserve zone id><br>
Product Type: <TMNSR|TMOR><br>
<table border=0>
<table of submitted data.>
...
<repeated for additional product types | reserve zones>
Response data for unsuccessful upload contains:
<td valign=top>
<center> Submission Failed: <br>
==============================<br>
<br>
Customer ID: <submitter's customer id><br>
Auction Start Date: <Auction Begin date><br>
Auction End Date: <Auction End date><br>
Reserve Zone ID: <reserve zone id><br>
Product Type: <TMNSR|TMOR><br>
<table border=0>
<table of submitted data.>
</table>
<Reason(s) for submission failure>
Because the SMS applications support the agent feature, where
the client may specify the represented party after having
authenticated, there is no need to use "third party" certificates
that simulate an identity within another company. We recommend
using a single certificate for all interactions with the SMS
applications, and using the agent feature to select the company
being represented. This makes certificate management easier and
provides a tighter audit trail for all of your interactions with the
SMS. CS&T can authorize your use of the SMS agent feature if you
need to represent multiple companies.
3.2 Client certificate storage
It is important to protect client certificates as strongly as
possible, because they represent your identity. Since your HTTPS
client program must have access to a client certificate, certificate
storage becomes an important management issue. Storage depends on
how the HTTPS client program will be run.
In a totally automated system, the HTTPS client program should probably be only run as a privileged system user (like "root" on a Unix machine). The client certificate (including the private key) would be stored in a file on disk that's readable only by that privileged user. If your HTTPS client requires certificate files to be password-protected (like Sun's JSSE for the Java platform), the password should be kept in a file on disk with the same permissions. The HTTPS client program can then read the password from this file when it attempts to access the certificate file. The password should never be present in configuration files or passed on the command line.
In a GUI front end, the HTTPS client program should probably use a
different method for obtaining the certificate, such as reading it
from a user's key store in a default location and prompting the user
for the password interactively. For the Java platform, for
instance, your GUI could use Sun's JAAS API to initiate
authentication with the user, depending on a JAAS authentication module that
uses JSSE to access a private key in the user's Java Keystore.
4. Implementation notes
4.1 Java: JSSE and java.net
A Java HTTPS client based on Sun's JSSE will be adequate
for accessing the SMS web applications. JSSE is an extension in
J2SE 1.3, and standard in J2SE 1.4, therefore it's easiest to
program the client for the J2SE 1.4 platform. JSSE provides SSL and
the HTTPS implementation, but the program can be written solely
against java.net APIs. The JSSE only has to be configured via
system properties and on the CLASSPATH at runtime.
Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());
java.net.URL.openConnection.
Because the protocol handlers property includes the JSSE HTTPS
implementation, an HTTPS URL will cause openConnection
to return a com.sun.net.ssl.HttpsURLConnection.
From Netscape 4.x, do the following to export your client certificate as a PKCS-12 file:
From the Windows desktop, do the following to export this CA certificate as a CER file:
A command line like the following can be used to import the CA certificate into your JRE's default keystore:
keytool -import -alias "Equifax Secure Certificate Authority" \
-file c:\temp\equifax-secure-cert-auth.cer \
-keystore <JAVA_HOME>\jre\lib\security\cacerts
The command line will prompt for the password mentioned earlier, and
you will have to answer "yes" to add the certificate to your trust
store.
The server should return the appropriate HTTP response code to the first HTTPS request. If the server does not return an HTTP response at all to the first request, then something in the network or SSL layers is failing. ISO IT can provide SSL error messages from our servers if it helps diagnose SSL problems.
If the client gets an HTTP response of 401 Unauthorized, the server either does not recognize the client's certificate at all or does not have an authorization for this certificate holder to access the requested URL.
| (Note: ISONE IT is investigating which situations will cause the web server to return 403 Forbidden instead of 401.) |
The most common cause of this is the JSSE system properties not being set properly, particularly the "javax.net.ssl.keyStore" property identifying your client certificate file. Re-run the client with the "-Djavax.net.debug=ssl" option to get more output and look for a "keyStore is :" message so you can check the settings you provided.
The most frequent cause of this is using JSSE with a client certificate that is not password-protected. Try re-exporting your certificate to a PKCS-12 file with a password and make sure your HTTPS client program provides that password using the "javax.net.ssl.keyStorePassword" system property.
This is the server complaining that something in your client certificate chain has expired. View the details of the client certificate in your .p12 export file and make sure each certificate in its chain is still valid. (The easiest way to do this is to go back to your browser and use its UI for viewing certificates.) This includes your user certificate, the intermediate signing certificate (VeriSign OnSite), and the root CA certificate (VeriSign). If the user certificate has expired, please export a current certificate or request a new one from ISONE. If the intermediate certificate or root CA certificate have expired, please contact ISONE for instructions on how to update them and produce a .p12 file that will work.
Re-run the JSSE-based HTTPS client with the "-Djavax.net.debug=ssl" option to get more output. You should get output like:
main, READ: SSL v3.1 Handshake *** ServerHello, v3.1 ... main, READ: SSL v3.1 Handshake *** Certificate chain ... ***The output following the "*** Certificate chain" line should indicate our smd.iso-ne.com certificate. Following the second "***" line is likely to be information about the problem with the certificate.
If it's followed by "issuer != subject DN" and "main, SEND SSL v3.1 ALERT: fatal, description = certificate_unknown", then the CA certificate is probably missing from your HTTPS client's trust store. Double-check the trust store contents for our CA certificate (currently "Equifax Secure CA").
If it's followed by "out of date cert:" and a dump of smd.iso-ne.com certificate, then our web server is sending you a bad server certificate. First make sure you are contacting the correct web server (for instance, smd.iso-ne.com instead of sandboxsmd.iso-ne.com). If you are contacting the correct one, please report the problem to ISONE so we can check our server id certificates, which may have expired.
Our web server is sending your client a server id certificate that does not match the requested URL. Run the client with the "-Djavax.net.debug=ssl" option to get more output. The second "main, READ: SSL v3.1 Handshake" line should be followed by a "*** Certificate chain", after which our server id certificate is dumped. Check the address in the Subject field and compare it to the address in the requested URL. Change your URL if you're accessing the wrong address. Otherwise, please report the problem to ISONE so we can check our server id certificates, which may have been placed on the wrong web servers.