Signature Method

The Signature Method

The next step involved in creating the digest is tracking and specifying the actual method used to create the signature (denoted by the <> element). After the canonical version of the XML is derived, the data that is part of the <> element desires to be converted into the actual signature value (and placed in the "SignatureValue" element). The "SignatureMethod" element specifies the algorithm that will be used for this operation.
The algorithm which is used to create the signature and, finally, the signature itself, are specified in the "SignatureMethod" tag and "SignatureValue" tag:

In this example segment, when the receiver gets the message, the signature is decrypted using the sender’s public key, the verified digest, and by verifying the sender’s signature. Who has to provide the Key information? In the following listing, the " KeyInfo " element holds the decryption key:

Here note that the XML signature doesn’t address trust of such key information. Then it is responsibility of whom? Generally, the application has to determine how trustworthy the key is. But for there is another way to verify that the supplied decryption key does belong to the sender, there is little point to the process. Anyone could intercept the message, change its contents, regenerate a public/private key pair, and re-sign the document. This will assert the public key belongs to the sender. This is the place; where the digital certificates come into the picture.
The certificate contains the binding between the identity of the public key’s owner and the key itself. For example, if the is omitted, the recipient is likely to identify the key that will be used, based on the application context. This type of issue is addressed in the XKMS specification, which is discussed in the later. Using XKMS or another PKI infrastructure, the recipient of the message can obtain the digital certificate, extract the public key from it, and verify that this key does belong to the sender.

The Signature Method

The Signature Method

The next step involved in creating the digest is tracking and specifying the actual method used to create the signature (denoted by the < SignatureMethod > element). After the canonical version of the XML is derived, the data that is part of the < SignedInfo > element desires to be converted into the actual signature value (and placed in the "SignatureValue" element). The "SignatureMethod" element specifies the algorithm that will be used for this operation.

The algorithm which is used to create the signature and, finally, the signature itself, are specified in the "SignatureMethod" tag and "SignatureValue" tag:

In this example segment, when the receiver gets the message, the signature is decrypted using the sender’s public key, the verified digest, and by verifying the sender’s signature. Who has to provide the Key information? In the following listing, the " KeyInfo " element holds the decryption key:

Here note that the XML signature doesn’t address trust of such key information. Then it is responsibility of whom? Generally, the application has to determine how trustworthy the key is. But for there is another way to verify that the supplied decryption key does belong to the sender, there is little point to the process. Anyone could intercept the message, change its contents, regenerate a public/private key pair, and re-sign the document. This will assert the public key belongs to the sender. This is the place; where the digital certificates come into the picture.

The certificate contains the binding between the identity of the public key’s owner and the key itself. For example, if the is omitted, the recipient is likely to identify the key that will be used, based on the application context. This type of issue is addressed in the XKMS specification, which is discussed in the later. Using XKMS or another PKI infrastructure, the recipient of the message can obtain the digital certificate, extract the public key from it, and verify that this key does belong to the sender.

XKMS

XKMS

Key Management
XKMS Structure

XML Key Information
XML Key Registration
Key retrieval
Location service
Validate Service
Key recovery
Key revocation
Key registration
Java Toolkits
Single-sign-on
Guidelines for signing XML documents

Key Management

Key Management

Keeping the public and private keys, digital signatures, and digital certificates organized and ecure is one of the biggest challenges for deploying all these new encryption, digital signature, and authentication technologies. Hence the need for a methodology for the management of the security components has been raised. In this progression, the XML Key Management ecification (XKMS) is been an emerging effort under the backing of the W3C. The goal of XKMS is to rovide standardized XML-based transaction definitions for the management of uthentication, Cryption, and digital signature services. The previous section discussed about the XML Encryption and XML Digital Signature specifications. However, these specifications assume that the web service responsible for processing the XML exists in an environment where keys and certificates are kept safe and secure.

The assumption here is that the web service programmer is aware of which certificates and
keys to use. XKMS will provide a set of XML definitions to allow developers to contact a third party. They will be helpful in locating and providing the appropriate keys and certificates.

The usefulness for allowing a third party to do this confidential job is to free the web service programmer from having to track the availability of keys or certificates and ensure their validity.

XML AND WEB SERVICES NOTES

In other words, XKMS will provide a standardized set of XML definitions to do the
following:

• Allowing developers to contact and use remote trusted third-party services
• The trusted third-party services will provide the following services:

encryption and decryption services

creation of keys

management of keys

authentication of keys and digital signatures

The specification standards specify a set of tags which is used to query external key anagement and signature validation services. For example, to know about the authentication of the ertificate, a client might ask a remote service to answer questions such as, “Is it a valid tificate?” or, “Provide the value of the key managed by you. ” Thus the facility to manage the keys is provided in XKMS.

XKMS was submitted to the W3C by Microsoft, VeriSign and web-Methods and is backed by a range of companies like HP, IBM Lenova etc. Thus XKMS is one of the three W3C specifications that define the XML security architecture.

XKMS Structure

XKMS Structure

On the whole the XKMS specifies the protocols for distributing and registering public keys. This is suitable for use in conjunction with the planned standard for XML signature and as an dditional standard for XML encryption.

The structure of XKMS contains two sections:

• XML Key Information Service Specification (X-KISS)
• XML Key Registration Service Specification (X-KRSS)

Let us explore the sections in detail.

XML Key Information Service Specification
XML Key Registration Service Specification

XML Key Information

XML Key Information Service Specification

X-KISS characterizes a protocol for a reliance service. It helps in managing the public-key information contained in documents that confirm to the XML signature specification. The basic objective of this protocol design is that relieving the XML programmers from the complex task of writing the code to process the XML signature ds:KeyInfo element. Essentially PKI may be ased upon a different specification such as X.509, the international standard for public-key certificates or Pretty Good Privacy (PGP), the widely available public key encryption system. Any trust policy can be utilized along with the XML signature specification.

When ever, a person is signing a document it is not necessary to specify any key information except that the value for the element . The value includes the key name, certificate name, key identifier and so on. Otherwise a link may be provided to a location which contains the required KeyInfo details.

XML Key Registration

XML Key Registration Service Specification

The Registration of the public key information is done through the protocol X-KRSS specifies. Once the key is registered it can be used along with other web services. The same protocol may be also used for recovery of the private keys. Since the protocol provides for authentication of the applicant, the key pair public key and private key may be generated by the applicant. This is the proof of possession of the private key. A means of communicating the private key to the client is provided if the private key is generated by the registration service.

The following section explains the key retrieval, location service and validates service with some example XML documents:

Key retrieval
Location service
Validate Service

Key retrieval

If the client wants the decryption key from a remote source, XKMS provides a simple method. Using the tag inside the element which is available in the XML signature can be used for this. The following segment assumes that a service exists that can provide information about a given key.
"KeyInfo"
"RetrievalMethod
URI=”http://www.KeyFil.samp/ValidateKey”
Type=”http://www.w3.org/2009/01/xmldsig#X509Certificate”/"
"/KeyInfo"

This search for a key is very simple and does not require the service to enforce the
validity of the key it returns.

Location service

If the application client wants to query a service for public key information then there are some set of tags available in the location service. If a web service client wants to encrypt something based on the value of the recipient’s public key, then the web service client should know the key value. For this requirement, it has to contact the key location service to obtain that key.

The following listing shows the

"Locate ",

"Query", and

"Response " tags used in the request:

Validate Service

The correspondence between the key and an attribute should be validated. Here the Validate Service facility available through a trusted third party can be used to get the job done. That third party validates the binding between a key and an attribute. For instance, look into the following query:

Key recovery

Key registration

How to register your key information with a third-party KMS?


Key revocation

How to send a request to the third-party KMS to tell it that you no longer want it to
manage the key on your behalf?

Key recovery

If you forgot your private key, then what to do? XKMS gives some solutions to this.
It describes how to send a request to obtain the private key and what the response looks
like. The specification does not state the rules under which the private key should be
returned. For example, it may be the policy of the service to cancel the old key and issue
a new one after certain period. However, that decision is up to the policy of the individual
provider.

Verisign is one of the primary drivers of XKMS. They have already released a Java
toolkit that supports XKMS development.

Key revocation

Key registration

How to register your key information with a third-party KMS?

Key revocation

How to send a request to the third-party KMS to tell it that you no longer want it to
manage the key on your behalf?

Key recovery

If you forgot your private key, then what to do? XKMS gives some solutions to this.
It describes how to send a request to obtain the private key and what the response looks
like. The specification does not state the rules under which the private key should be
returned. For example, it may be the policy of the service to cancel the old key and issue
a new one after certain period. However, that decision is up to the policy of the individual
provider.

Verisign is one of the primary drivers of XKMS. They have already released a Java
toolkit that supports XKMS development.

Key registration

Key registration

How to register your key information with a third-party KMS?

Key revocation

How to send a request to the third-party KMS to tell it that you no longer want it to
manage the key on your behalf?

Key recovery

If you forgot your private key, then what to do? XKMS gives some solutions to this.
It describes how to send a request to obtain the private key and what the response looks
like. The specification does not state the rules under which the private key should be
returned. For example, it may be the policy of the service to cancel the old key and issue
a new one after certain period. However, that decision is up to the policy of the individual
provider.

Verisign is one of the primary drivers of XKMS. They have already released a Java
toolkit that supports XKMS development.

Java Toolkits

IBM XML Security Suite and the Phaos XML Toolkit are some of the JAVA Toolkits
for XML security available. The toolkits use Xerces and Xalan to parse the XML data.
The assembly of signatures is done by using their own APIs. The same is used for encrypting
the data. The Phaos sample simply used parser APIs such as
doc.getElementsByTagName(tagName) to access the element to be encrypted, as shown
in the following listing:

// Copyright © Phaos Technologies
public class XEncryptTest
{
public static void main (String[] args) throws Exception

{

... // usage, command line args...

// get the XML file and retrieve the XML Element to be encrypted

File xmlFile = new File(inputFileName);

DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();

dbf.setNamespaceAware(true);

DocumentBuilder db = dbf.newDocumentBuilder();

Document doc = db.parse(xmlFile);

Element inputElement = null;

NodeList list = doc.getElementsByTagName(tagName);

if (list.getLength() != 0)

inputElement = (Element) list.item(0);

else

{

System.err.println(“XML element with tagName “ + tagName + “ unidentified.”);

System.exit(1);

}

// Create a new XEEncryptedData instance with the owner

// Document of the input xml file,the data type URI and

// the Id “ED” for this EncryptedData element.

XEEncryptedData encData = XEEncryptedData.newInstance(doc, “ED”, dataType);

... // determine encryption algorithm

// set up the EncryptionMethod child element

XEEncryptionMethod encMethod = encData.createEncryptionMethod(algURI);

encData.setEncryptionMethod(encMethod);

// set up the symmetric key to be used in encryption

SymmetricKey key = null;

File keyFile = new File(keyFileName);

... // File stuff

// set up the ds:KeyInfo child element with the keyName

XSKeyInfo keyInfo = encData.createKeyInfo( );

keyInfo.addKeyInfoData(encData.createKeyName(keyName));

encData.setKeyInfo(keyInfo);

// set a nonce value to be prepended to the plain text

byte[] nonce = new byte[16];

encData.setNonce(RandomBitsSource.getDefault().randomBytes(nonce));

// encrypt the XML element and replace it with the

// newly generated EncryptedData element

System.out.print(“Encrypting the XML data ... “);

XEEncryptedData newEncData = XML AND WEB SERVICES NOTES

XEEncryptedData.encryptAndReplace(inputElement, key, encData);

System.out.println(“done”);

// output the XML Document with the new EncryptedData element to a

// file

}

}

The Phaos toolkit was much easier to set up and run than the IBM toolkit. This piece
of makes a call to encryptAndReplace( ). This method takes the element that we’ve given
it, encrypts it by using the given key, and replaces the original element with the appropriately
tagged, encrypted element.

As a whole, it can be said that Web services security is still an emerging area and proper
handling of this portion has to be done by researchers and vendors together.

Single-sign-on

What is this Single-sign-on? It is nothing but the ability for an end user or application
to access other applications within a secure environment. It has to be done without needing
to be validated by each application. The most common example of single-sign-on technology
is in web-based corporate intranet applications.

What is the use of this environment? In this setting, the users may want to use various
applications that allow access to their timetable, Project schedule, expense reports and
health benefits. If each user of the application need to be authenticated individually then the
following may occur such as in convenienence, slow, and limiting the value of the intranet
site. The single sign on is one of the solution which allows access to all applications without
additional intervention after the initial sign on, using a profile that defines what the user is
allowed to do.

Many companies provide products for web-based, single-sign-on authentication and
authorization, including companies such as Netegrity, Securant (now a part of RSA), Oblix,
and Verisign. These products with the help of an intermediary process which controls and
manages the passing of user credentials from one application to another. Users are assigned
a permit that carries their rights information and simultaneously allows them to access
many applications without the need to authenticate each one. This permit allows applications
within the secure environment to shift the burden of authentication and authorization to a
trusted third party, leaving the application free to focus on implementation of business
logic.

The single-sign-on concept is easily extended to web services. Web services can be
given a permit (placed in an XML/SOAP message) that can be used to validate the service
with other web services. However, the secure use of web services will depend on the
ability to exchange user credentials on a scale never seen before. Individual services will
reside in a variety of protected environments, each using various security products and
technologies. Providing a way to integrate these environments and enable their interoperability
is critical for the secure and effective use of these services.

Based on XML, the Security Assertion Markup Language (SAML) is an almost
complete specification proposed by the Organization for the Advancement of Structured
Information Standards (OASIS). The primary goal of SAML is to enable interoperability
between different systems that provide security services. The SAML specification does
not define new technology or approaches for authentication or authorization. Rather, it
defines a common XML language that describes the information or outputs generated by
these systems.

Guidelines for signing XML documents

Signing of XML documents needs care, since any change in the document like
introduction of white space, change of case tend to change the signature.

The following two points to be kept in mind when going for signing the document:

1. Content Presentation techniques may introduce changes
2. Transformation may alter the content

XML relies on transformations and substitutions during the processing of XML
documents. For example, if an XML document includes an embedded style sheet or
references to an external style sheet, the transformed document should be represented to
the user rather than the document without the style sheet. In this case, the signer should be
careful to sign not only the original XML but also the other information that may affect the
presentation.

While due consideration is not been given for handling the original and transformed
document, it will return a different result than intended. As in any security infrastructure,
the security of an overall system will depend on the security and integrity of procedures
and personnel as well as procedural enforcement.

XML - INDEX

UNIT III

XML-RPC

Data Typing ZwiftBooks and XML-RPC

XML-RPC Responses


SOAP


Road to SOAP


HTTP


OVERVIEW OF SOAP



SOAP Protocol


SOAP Overview



SOAP Message Structure



SOAP Messaging Example



SOAP and Actors



SOAP Design Patterns



SOAP Faults



SOAP with Attachments


The W3C and SOAP



UNIT IV

What Is Web Services


Web Services - A ZwiftBooks Perspective


Web Services Technologies



Key Technologies


UDDI

UDDI Failure and Recovery

WSDL


From Abstraction to Reality


XML Index


ebXML


UN/CEFACT and OASIS are key players behind ebXML.


ebXML Technologies


The technical architecture consists of several pie


.NET, J2EE, and Beyond


Transactions


NET and J2EE


Sun ONE and Web Services


.NET


The .NET Platform

The .NET Framework

J2EE

Object Oriented Programming

SCRIPTING LANGUAGE

XML-RPC

• XML-RPC, which does remote procedure calls over the Internet, is a great example of out-of-the-box thinking.

• In confronting the communication problem of how a program on machine A can get some code on machine B to run, XML-RPC ignores the difficulty entirely and delegates the transport to HTTP, focusing instead on the details of what to say, not how to get the message there.

• Early work on XML-RPC was done by Dave Winer of UserLand Software.



Winer had been working on one of the classic problems of distributed computing:

• how to get software running on different platforms to communicate.

• Shortly after XML came out in 1998, Winer demonstrated cross-platform communication by placing XML remote procedure commands in the body of an HTTP POST request.

• Because XML-RPC depends on HTTP to move data from one server to another, it only needs to define an XML vocabulary that specifies the name of some piece of code to execute remotely and any parameters the code might need.

Data Typing ZwiftBooks and XML-RPC

Data Typing

• In keeping with the spirit of Web reuse, XML-RPC uses XML Schema data types to specify the parameter types of the procedure call.

• Data types include scalars, numbers, strings, and dates, as well as complex record and list structures.

ZwiftBooks and XML-RPC

• To allow other computer systems to query the ZwiftBooks server about the availability and delivery time of a badly needed book.

• The basic idea is that the user supplies an ISBN and a zip code and ZwiftBooks returns the guaranteed delivery time.

• Figure illustrates the use of XML-RPC over HTTP to trigger the execution of a procedure called getGuaranteedDeliveryTime based on ISBN number and zip code.






The XML-RPC specification places a number of minimal requirements on the XML, including the following:
• The XML payload must be well-formed XML and contain a single methodCall structure.

• The methodCall element must contain a methodName sub-item consisting of a string that names the method to be called.

• If parameters are required, the methodCall element must contain a params sub-item that contains individual param elements, each of which contains a single value.

XML-RPC Responses

• The job of the server is to process the XML-RPC request for the execution of some piece of code and return a value to the client.

• According to the rules of XML-RPC, a server must return either the result of the procedure execution or a fault element.

• Figure also illustrates the return value of an XML-RPC packaged in the data area of an HTTP reply. Again, as far as HTTP is concerned, it's just data.

• XML-RPC specifies that the response to a procedure call must be a single XML structure, a methodResponse, which can contain either the return value packaged in a single params element or a fault element which contains information about why the fault occurred.

• Figure also illustrates returning a fault element as the payload of an HTTP response.

• As we'll see with SOAP, the specification for describing failure is an important aspect of XML-based protocols.

SOAP

• XML- used in defining industry-specific data description languages
-used as a protocol language that has enabled communication and data exchange across the Web.

• XML has proven effective in promoting data exchange between partners and collaborators across a wide range of industries,

• A new perspective on middleware with XML protocols such as XML Remote Procedure Call (XML-RPC) and Simple Object Access Protocol (SOAP), - offer platform, language, and transport independence for data exchange between partners and suppliers.

• Transporting XML - HTTP request- Web-based distributed computing and the emergence of Web services.


What Is SOAP and Why Is it Important?

• SOAP is an XML-based protocol for exchanging information in a decentralized, distributed environment.

• Made for the Web, a combination of XML and HTTP that opens up new options for distributed data exchange and interaction in a loosely coupled Web environment.

• SOAP is a technology that allows XML to move easily over the Web.

• SOAP does this by defining an XML envelope for delivering XML content and specifying a set of rules for servers to follow when they receive a SOAP message.



• SOAP has opened opportunities for extending the enterprise.






• Before SOAP- two options for moving data between partners.

Option-I
• To build a wide area network spanning a broad geographic region and let partners plug into it.

• Approach taken by Electronic Data Interchange (EDI), which defined messages and protocols for data transfer but left the network details up to the partners.

• Result- collection of networks that pretty much locked the partners in and made it difficult and expensive to reach out to other EDI networks and costly to bring in new partners.



Option-II
• Approach for moving data between partners was to build a distributed object infrastructure than ran over the Internet.

Common Object Request Broker Architecture (CORBA), Remote Method Invocation (RMI), and Distributed Component Object Model (DCOM).

• The problem was that each had to decide on a protocol that could sit on top of TCP/IP and handle interobject communication.

• CORBA chose Internet Inter-ORB Protocol (IIOP), DCOM chose Object Remote Procedure Call (ORPC), and RMI chose Java Remote Method Protocol (JRMP).

• Drawback was that CORBA could talk to CORBA, RMI to RMI, and DCOM to DCOM, but they could not talk to each other nor directly to the Web except through special sockets


SOAP is one of several options for moving data across the Web.









Option-III
• SOAP, combines the data capabilities of XML with the transport capability of HTTP,

• Overcoming the drawbacks of both EDI and tightly coupled distributed object systems such as CORBA, RMI, and DCOM.

• It does this by breaking the dependence between data and transport and in doing so opens up a new era of loosely coupled distributed data exchange.


Inception in 1998, SOAP has gained wide acceptance across the software industry.
Its impact is evident from the following observations:

• Web services frameworks use SOAP as the transport technology for delivering data and XML-RPC messages across distributed networks.

• Microsoft is committed to SOAP as part of its .NET initiative.

• Sun is using SOAP in its Sun Open Net Environment (Sun ONE) Web services framework.
• IBM, which has played a major role in the SOAP specification, has numerous SOAP support tools, including a SOAP toolkit for Java programmers.

• IBM has donated the toolkit to Apache Software Foundation's XML Project, which has published an Apache-SOAP implementation based on the toolkit.

• CORBA Object Request Broker (ORB) vendors such as Iona are actively supporting SOAP in the form of CORBA-to-SOAP bridges.

Road to SOAP

The Road to SOAP

• As far back as the 1960s, companies turned to computer automation to reduce the paperwork burden associated with purchase orders, bills of lading, invoices, shipping orders, and payments.

• Driven by a need to standardize the exchange of data between companies doing business with each other, in 1979 the American National Standards Institute (ANSI) chartered the Accredited Standards Committee X12 (ASC X12) group to develop uniform standards for interindustry electronic interchange of business transactions.

• The result was a collection of standards known as the Electronic Data Interchange, better known today as EDI.




• As Figure shows, EDI is built around point-to-point networks that require partners to use software that implements EDI's data and messaging specifications.

• It is expensive both to develop and to maintain. In addition, once an EDI system is in place, changes must be agreed upon and implemented by all participants.

• For medium- and small-size businesses, EDI's cost is prohibitive.

• There will always be a need for a WAN wrapper, a network over which to deliver the data.

• Using the Internet as the global WAN wrapper and XML as the data format, the problem of data distribution is greatly simplified. The missing piece is how to get the data from point A to point B, which leads us to HTTP.

HTTP

• HTTP is an important building block for using XML as a Web-based messaging protocol.

• Although the Internet and various protocols such as FTP and TELNET had been in existence since the 1970s for moving files, sending email, and allowing individuals to connect remotely, it wasn't until 1992 that the face of the Internet was changed through the use of a simple request-response protocol known as HTTP.

• Figure shows that HTTP works much like FTP except that the contents of a file are delivered to a browser instead of a filesystem.



• EDI works by providing a collection of standard message formats and element dictionaries so that businesses can exchange data using networks of their choice.

• EDI's early success in the transportation industry led to its adoption by other industries, including health care insurance, management, financial services, and government procurement.

• Over the past two decades over 100,000 organizations have used EDI to conduct business with partners and suppliers.

• EDI suffers from the same problem faced by all pre-Web, tightly coupled technologies: network lock-in.



Both HTTP and FTP move data across the Internet. FTP delivers data directly to disk while HTTP delivers it to a browser. When the data is in HTML or a format the browser understands, we have the Web




• To understand how XML is used as a protocol language it is instructive to take a look at how HTTP works. The first HTTP specification written by Tim Berners-Lee is a study in simple elegance

• Clients request files from servers using a simple text string of the form:
GET filename

• This command is interpreted as a request to a server listening on port 80.
• The response of the server is either the contents of the requested file or a string indicating an error

• HTTP gains its power from its simplicity and its explicit avoidance of transport lock-in.

• HTTP sits on top of TCP/IP, which is responsible for reliably moving data between Internet nodes.

• HTTP, a simple request-response Web protocol, has been the catalyst for XML's widespread use.

• The HTTP GET command requests a Web page. The HTTP POST command delivers information and receives information back.





POST Me Some Data

• The POST command is a request for a server to do something with data delivered as part of the POST message.

• POST was included in the HTTP specification in order to deliver HTML form data to a server for processing by some server program.

• The structure of a POST request is similar to a GET, except that data intended for the server appears after the header and is referred to as the body or payload of the request.



• Figure illustrates the structure of an HTTP request showing the difference between GET and POST.

• When a POST request arrives at a server, the server looks for data following the blank line that signals the end of header information.

• This data delivery mechanism turns out to be the key element in moving XML across the Internet.

• Instead of supplying data from an HTML form, the payload slot of an HTTP request can just as easily be packaged with XML.


• The structure of an HTTP request provides an opportunity for delivering XML.

• As far as HTTP is concerned, it's just data.




• As Figure 4.6 shows, XML's transport independence means that it may be carried by any Internet protocol, including HTTP and FTP, or even sent via mail using Simple Mail Transfer Protocol (SMTP). This freedom to move data has opened the door to XML-RPC, SOAP, and the entire Web services initiative.

• XML and HTTP are loosely coupled, with no internal dependencies on each other. Distributed infrastructures such as CORBA, RMI, and DCOM are tightly coupled, with dependencies between data and transport

OVERVIEW OF SOAP

SOAP

SOAP carries on the XML-RPC tradition by defining an XML language for packaging arbitrary XML inside an XML envelope.

HTTP will usually be used as the transport protocol for SOAP messages.

SOAP is important for seeing how XML can be used to move information across the Web and how it fits in the grand vision of making XML-based distributed computing a reality.

SOAP Background

• The SOAP 1.0 specification was developed by Microsoft, DevelopMentor, and Dave Winer of UserLand Software and released in the spring of 1998.

• Prior to the release of SOAP 1.0, Winer released his work on RPC as the XML-RPC specification http://www.xmlrpc.com, which is very close to SOAP 1.0.

• Following the release of SOAP 1.0, IBM and Lotus joined the original developers, along with a group of partners including Ariba, Commerce One, Compaq, IONA, Intel Corp., ObjectSpace, Rogue Wave, and others.

• SOAP 1.1 was published by the W3C as a Note in May 2000. In July 2001, the W3C released the first public Working Draft for SOAP Version 1.2 based on the work of the W3C's XML Protocol Working Group

• The SOAP specification and its influences

SOAP Protocol

The SOAP Protocol
• As Figure illustrates, SOAP is a transport protocol similar to IIOP for CORBA, ORPC for DCOM, or JRMP for RMI.

SOAP differs from CORBA, RMI, or DCOM in several ways:

• IIOP, ORPC, and JRMP are binary protocols, while SOAP is a text-based protocol that uses XML. Using XML for data encoding makes SOAP easier to debug and easier to read than a binary stream.

• Because SOAP is text-based, it is able to move more easily across firewalls than IIOP, ORPC, or JRMP.

• SOAP is based on XML, which is standards-driven rather than vendor-driven.

• Figure 4.9. The SOAP protocol opens up new options for data exchange across the Web






• The net effect is that SOAP can be picked up by different transport protocols and delivered in different ways.

• For example, when used with HTTP it can be delivered to a Web server; when used over FTP it can deposited directly into a file system; and when used with SMTP it can delivered to a user's mailbox

• Figure 4.10 illustrates that SOAP can be used for direct connection between sender and receiver, or, with the use of messaging middleware,

• SOAP messages can be stored for subsequent delivery and/or broadcast to multiple receivers.

• SOAP extends the Web from server-to-browser to server-to-server interaction.




• Many companies, using SOAP as protocol for exchanging data between established partners is proving a totally satisfactory way to leverage the benefits of XML and the Web.
• All that is required is an agreed-upon schema, either a DTD or an XML Schema, for the XML data being exchanged and a SOAP server capable of handling the incoming XML as it arrives over the Web.

• Details about what kind of schemas to expect and who will check that the XML conforms to the schemas are decided offline by individuals participating in the process.

• On the software side, senders need to be involved in packaging their data in an XML document.

• For those companies already storing data in XML, this should require only minimal effort.

• If the stored XML data is not in the form required by the agreement, an XSL Transformations (XSLT) style sheet can be programmed to automate the transformation.

SOAP Overview

SOAP consists of three parts:

•Encoding rules that control XML tags that define a SOAP message and a framework that describes message content

•Rules for exchanging application-defined data types, including when to accept or discard data or return an exception to the sender

•Conventions for representing remote procedure calls and responses

•SOAP messages define one-way data transmission from a sender to a receiver. However, SOAP messages are often combined to implement patterns such as request-response. When using HTTP bindings with SOAP, SOAP response messages can use the same connection as the inbound request

SOAP Message Structure

• Figure 4.11 illustrates the structure of a SOAP message, consisting of three parts:

• The SOAP Envelope: The outermost element of a SOAP message. The Envelope element is the root of the XML document that defines a SOAP message.

• The SOAP Header: The Header element provides a modular way of directing SOAP servers to do processing before passing the SOAP message on.

• For example, it is possible to add SOAP header information that instructs a server to add transaction or authentication information.

• Headers are also important in building piped architectures where processing is done in stages and data is modified as it is passed from handler to handler.

• The SOAP Body: An element that must appear in a SOAP message. The Body element is where the transported XML is loaded. SOAP makes no assumptions about the kind of XML transported in the body of a SOAP message. The data may be domain-specific XML or it may take the form of a remote procedure call.



• Figure 4.11. SOAP messages have a common format that includes a SOAP Envelope, an optional Header, and a Body section that contains the message content. SOAP also defines a message path and set of roles that SOAP nodes can adopt along a path.

SOAP Messaging Example

• In the previous section we looked at sending XML-RPC over HTTP to execute specific procedures on our ZwiftBooks server. With SOAP we may still use XML-RPC to trigger specific methods on the server, or we may simply define XML elements that get processed by our SOAP server.


• To understand how SOAP works, let's continue our ZwiftBooks example and look at how SOAP may be used to expand business functionality by opening our server up to collectors who wish to notify ZwiftBooks about books they have for sale. ZwiftBooks will then add the providers and their books to the ZwiftBooks database.


• To make this happen, several things must be done by ZwiftBooks:


• Define a top-level element and related subelements that will trigger processing of the book availability data by the SOAP server.


• Define a schema (DTD or XML Schema) that dictates the form of the XML that will arrive from collectors and book providers.


• Specify a namespace that is unique to ZwiftBooks. This may be the ZwiftBooks Web site or any URI.


• Configure the server to return a fault if the incoming SOAP message is not one of the special elements defined in step 1.


• Figure 4.12 illustrates a SOAP request for the ZwiftBooks guaranteed delivery time for a specific book, specified by ISBN number. Note that the SOAP Envelope element is the top-level element in the body of the SOAP message and within the SOAP body element is found the request for GetGuaranteedDeliveryTime, packaged as an element including the ISBN number.


Figure 4.12. A SOAP request sent to the ZwiftBooks server.




• A SOAP response takes the form illustrated in Figure 4.13. Here the XML response indicating the best delivery time is packaged in a SOAP message that is delivered using the standard HTTP response protocol. As far as HTTP is concerned, it's just data being returned to the client that initiated the request. However, for a client that understands SOAP, the data becomes useful information.

Figure 4.13. A SOAP response to a request to the ZwiftBooks server.





Message Paths

• An important aspect of SOAP is the provision for message paths. Independent of the transport protocol used to send SOAP messages, messages may be routed from server to server along a so-called message path. As we saw in Figure 4.11, message paths support message processing at one or more intermediate nodes in addition to the ultimate destination.

SOAP Intermediaries

• SOAP intermediaries are an essential aspect of building scalable Web-based distributed systems. Intermediaries can act in different roles, including proxies, caches, store-and-forward hops, and gateways. Again, experience with HTTP has shown that intermediaries cannot be implicitly defined but must be provided as an explicit part of the messaging path model. Thus, one of the key motivations of the Working Group is to ensure that an XML protocol supports composability between peers.


• A SOAP-compliant server must be able to act as a SOAP intermediary capable of processing and forwarding a SOAP message on a path from its origin to a final destination. SOAP intermediaries may be explicitly specified by providing their URIs as the value of the SOAP actor attribute within a SOAP header, for example:


• Alternatively, a SOAP message can specify that the header should be processed by the first SOAP application that processes the message, as in this example:

SOAP and Actors

• If the SOAP actor attribute is not present in a header, then the recipient of the message is considered the final destination. The following list covers the rules a well-behaved SOAP server must follow when receiving a SOAP message:

• Identify the parts of the SOAP message intended for that application. This means checking the header for an actor attribute that is either the URI of the application or the URI http://schemas.xmlsoap.org/soap/actor/next, which means that the application must process the header.


• Verify that all parts of the header intended for the application and associated with a mustUnderstand="true" attribute are supported by the application. If the application cannot process the message, then it must discard the message and return a SOAP fault (see section on SOAP faults on page 137).


• Process the parts of the header intended for the application. If there are elements that include the attribute mustUnderstand="false" or that do not specify the mustUnderstand attribute, then the application may ignore those elements.


• If the application is not the ultimate destination of the message, then it must remove all header elements intended for it before forwarding the message.

SOAP Design Patterns

• Architectural design patterns capture the proven experience of software architects in the form of architectural patterns, each with distinct properties that address different architectural design problems. To a business strategist, software architecture patterns provide a high-level conceptual view of a software system independent of the specific implementation's details. To a software engineer, patterns provide a structure within which components can be designed and integrated. To those tasked with moving from requirements to design, patterns provide a starting point for developing implementation strategies.

• Figure 4.14 illustrates two of the architectural patterns in wide use today: the Layers Pattern, and the Pipe and Filter Pattern. The Layers Pattern is one of the earliest successful architectural patterns and is the basis for the design of telecommunications infrastructures such as TCP/IP. Layers describe a solution to a problem by partitioning the problem into subtasks that are assigned to individual layers. Each layer draws on services from the layer below and provides services to the layer above.

• Figure 4.14. Architectural design patterns: The Layers Pattern and the Pipe and Filter Pattern.

• A variant of the Layers pattern, the Multitier Pattern, is used to build tiered systems. For example, two-tier client-server systems partition display functionality to the client tier and business logic to the server tier. Three-tier systems add another tier for managing data.

• The Pipe and Filter Pattern is another architectural approach to systems construction that assumes a stream of data moving through a series of pipes and filters and being transformed as it travels from origin to destination. In addition to serving as transforming agents, stages in the Filter Pattern can be used to buffer data and help deal with issues that arise in working with interruptible network connections.

• The SOAP specification provides support for the filters through its ability to specify intermediary processing agents along a path from an XML document's origin to a final destination.

• Filters are also supported by the Simple API for XML (SAX), a Java application program interface for processing XML. SAX makes use of a filter class that can act as an intermediary in a complex chain of processing. The filter class is passed as the event handler to another class that generates SAX events, forwarding all or some of those events to the next handler or filter in the processing chain.

• Filters may be used to prune the document tree by not forwarding events for elements with a given name (or that satisfy some other condition), or a filter can generate new events to add parent or child elements to an existing document stream. Filters can also be used to add or remove element attributes or modify the character data.

• The Filter is related to UNIX's capability to take simple processing functions and pipe them together to perform complex tasks. The power of SOAP and SAX filters derives from the fact that simpler, easy-to-maintain filters may be chained together to produce complex XML data transformations

SOAP Faults

• SOAP faults occur when an application cannot understand a SOAP message or when an error occurs during the processing of a message. SOAP defines an XML fault element that carries error and/or status information back to the message sender.

• Faults are intended to provide detail to the sender as to why the fault occurred. The information that can be returned as part of a fault includes the following:

• faultcode: SOAP defines a set of faultcodes for basic SOAP errors, although an application may provide its own codes.

• faultstring: This element provides a readable explanation as to why the fault occurred.

• detail: The value of the detail element is that it provides information about the problem that occurred while processing the Body element. If not present, it indicates that the problem did not occur in the body of the SOAP message.

SOAP with Attachments

SOAP with Attachments

•SOAP provides a protocol to deliver XML across the Internet. However, requirements often dictate that not just XML needs to be transported but also other related documents such as DTDs, schema, Unified Modeling Language diagrams, faxes, public and private keys, and digests that may be related to the XML. In keeping with the spirit of the Web not to introduce new technologies when existing ones are available, SOAP relies on the existing rules for HTTP attachments to deliver auxiliary data with a primary SOAP message, allowing a SOAP message to reference the attachments.

•The SOAP with Attachments (see Figure 4.15) document defines a binding for a SOAP message to be carried within a Multi-Purpose Internet Mail Extensions (MIME) multipart/related message in such a way that the processing rules for the SOAP message are preserved. The MIME multipart mechanism for encapsulation of compound documents can be used to bundle entities related to the SOAP message, such as attachments

•Figure 4.15. SOAP with Attachments lets additional documents travel with SOAP-based XML content using HTTP as the transport protocol

SOAP and Firewalls

•SOAP's global reach is made possible by its alliance with HTTP, the Internet protocol that is the basis for moving data back and forth from Web servers to browsers. HTTP works by accessing Web servers on port 80, which is kept open for Web traffic. Most servers shut down other ports for security purposes.

•SOAP's use of port 80 is a double-edged sword. While an open port 80 makes SOAP messaging possible, it also makes system managers nervous about incoming SOAP traffic, since SOAP messages traveling on port 80 bypass the protection afforded by firewalls. SOAP messages can contain XML-RPC commands to execute code on the server, which requires caution to protect the server from unwanted attacks, the form of which is difficult to anticipate.

•It should be noted that while XML-RPC calls can easily pass through firewalls, XML-RPC distinguishes itself from other server traffic by including a header element that specifies content-type as text/xml. This at least alerts the server and associated firewall software that XML is being POSTed to the server.

The W3C and SOAP

The W3C and SOAP

• The XML Protocol Working Group is the W3C group formed in response to the submission of the SOAP 1.1 specification as the basis for a universal XML-based protocol. The formation of the Working Group signals the W3C's willingness to consider extending the Web from a network that delivers documents and links to human users, to a network that supports communication between applications.

• The goal of the XML Protocol Working Group is the creation of simple protocols that can be deployed across the Web and easily programmed through scripting languages, XML, and Web tools. It is important to note that the goal of the Working Group is not to provide a complete infrastructure for Web communication, but to build a foundational layer that can be incrementally extended to support the security, scalability, and robustness required for more complex applications.

• A key aspect of the XML Protocol Working Group is that, like all other W3C initiatives, the group effort must fit within the broader W3C goals of modularity and simplicity. In defining a protocol for the Web, it is important that the final version of the envelope and any serialization mechanisms developed by the Working Group should not preclude any programming model nor assume any particular mode of communication between peers. In addition, it should also support distributed extensibility where the communicating parties do not have a priori knowledge of each other.

• By limiting the scope of its effort to include neither transport nor application-specific features, the Working Group is better positioned to achieve its goal of producing a simple mechanism for encapsulating and representing data that is transferred between communicating peers. In keeping with the foundational W3C design principle of avoiding constraints if at all possible, the XML Protocol Working Group carries on the W3C philosophy of fostering interoperability.

• Much of the work on SOAP has been influenced by the experience of developing HTTP, which has demonstrated how difficult it is to retrofit support for evolution and how important extensibility is as a feature of an infrastructure

Taking SOAP to the Next Level

• Going beyond the simple use of SOAP to exchange data, several options are emerging that use SOAP as their base protocol.

As can be seen in Figure 4.16, other options include Electronic Business XML (ebXML) and Web services.

Although we examine ebXML and Web services in more detail in Chapter 5, it's important to realize that both these technologies impose some structure on the freewheeling world of SOAP-based communication.

As we'll see, ebXML is useful in defining messages and processes for common B2B transactions, and Web services is an infrastructure for discovering and connecting to services anywhere on the Web.

Thus rather than spending time and money defining a schema for purchase orders, a company can turn to ebXML or a Web services framework to provide a structure for communication. However, using SOAP alone is a completely satisfactory approach with minimal risk that gets the job done.

• Figure 4.16. SOAP offers an envelope for sending XML data across the Web. Technologies such as Web services and ebXML add structure and process to the B2B dialog.

What Is Web Services

Web services builds on SOAP and HTTP.

Web services is at once a technology, a process, and a phenomenon.

As a technology it is a set of protocols that builds on the global connectivity made possible by SOAP and the synergies of XML and HTTP.

As a process, it is an approach to software discovery and connection over the Web.
As a phenomenon, it's an industry-wide realization that the decentralized, loosely coupled, synergistic Web can't be ignored.

Web services builds on SOAP's capability for distributed, decentralized network communication by adding new protocols and conventions that expose business functions to interested parties over the Internet from any Web-connected device.

SOAP, for example, is not a stand-alone technology, but the result of synergies between XML and HTTP.




Web services is a technology and process for discovery and connection.

Web services represents an industry-wide response to the need for a flexible and efficient business collaboration environment.

Technically, it is a way to link loosely coupled systems using technology that doesn't bind them to a particular programming language, component model, or platform.

Practically, it represents a discrete business process with supporting protocols that functions by describing and exposing itself to users of the Web, being invoked by a remote user, and returning a response. 

It includes:

Describing: Web services describes its functionality and attributes so that other applications can figure out how to use it.

Exposing: Web services register with a repository that contains a white pages holding basic service-provider information, a yellow pages listing services by category, and a green pages describing how to connect and use the services.

Being invoked: When a Web service has been located, a remote application can invoke the service.

Returning a response: When a service has been invoked, results are returned to the requesting application.

The driving force behind Web services is the desire to allow businesses to use the Internet to publish, discover, and aggregate other Web services using the global underpinning of SOAP.

Web Services - A ZwiftBooks Perspective

Web Services: A ZwiftBooks Perspective

ZwiftBooks uses a Web services repository to list its offerings
Book delivery service, It must

Decide on the service it wants to provide

Pick a registry (or registries) for uploading its information

Decide how to list its service at the registry

Define explicitly how users can connect to its service


Deciding on a Service

The first step is to decide what aspects of the business ZwiftBooks wants to expose to potential clients.

Picking a Registry

There are currently several registries implemented by various software vendors.

ZwiftBooks selects SuperReg, a registry run by a major Web services company that is known to get lots of traffic.

Deciding How to List

An important issue for ZwiftBooks is how is get discovered on the registry.

Table outlines the options for storing information in a UDDI repository.

Web services registries support white, yellow, and green pages.

Of course, ZwiftBooks will be in the white pages under "Z,".

What ZwiftBooks needs is to list itself by category, just like in the conventional yellow pages. A logical place would probably be "Books." But is that enough?

Individuals and software agents will search repositories.

Remember, Web services is intended for computer-to-computer interactions.

There may be a human on the other end of the computer trying to find a book service,

ZwiftBooks needs to think about how to make itself attractive to computer-based agents trolling the Web for business partners.

Is ZwiftBooks prepared to deliver book information based on industry standards? If so, then maybe ZwiftBooks is worth a further look; if not, it's time for the robot to move on.

Web services yellow pages will list companies according to conformant standards.

What will be important in attracting software agents

Thus, instead of listing itself just under "Books" in the yellow pages, ZwiftBooks may also want to list itself as conformant to the International Book Exchange Consortium or whatever standards it is prepared to abide by.

Potential customers and partners can at least know in advance that it is possible to do business with ZwiftBooks according to their own processes and procedures.

Defining How to Connect

The final step for ZwiftBooks is to place an entry into the green pages that describes how to connect to and query the ZwiftBooks server.

To do this ZwiftBooks prepares a WSDL document using WSDL's XML syntax.

Operation
Publish: How the provider of Web services registers itself

Directory

White pages: Name, address, telephone number, and other contact information of a given business

Information

Business information: A businessEntity object contains information about services, categories, contacts, URLs, and other things necessary to interact with a given business

Operation

Find: How an application finds a particular Web service

Directory

Yellow pages: Categories of businesses based on existing (nonelectronic) standards

Information

Service information: Describes a group of Web services. These are contained in a businessService object.


Operation

Bind: How an application connects to and interacts with Web services after it's been found


Directory

Green pages: Technical information about the Web services provided by a given business


Information

Binding information: The technical details necessary to invoke Web services. This includes URLs, information about method names, argument types, and so on. The bindingTemplate object represents this data.

Service specification detail: This is metadata about the various specifications implemented by a given Web service. These are called tModels in the UDDI specification