Version 1.2
Authors C. Larsson, N. Wedi, H. Thiemann
The purpose of the PRISM system is to enable users to perform numerical experiments, coupling interchangeable model components using
standardised interfaces.
The general architecture provides the infrastructure for this. There is an emphasis on choosing an architectural design that allows these activities
to be done remotely, eg. without the user physically being in the place where the numerical computations take place.
Within the PRISM system three different types of actors exist:
Users: someone who uses the PRISM System to assemble and run a coupled model.
Model developer: someone who makes changes to a PRISM model code that will be introduced to the default model version.
Administrator: the person who provides the link between developers and users.
These different actors represent different demands on the system.
The best design in order to fulfill the PRISM requirements (remote access, modularity, extendability) is a centralised architecture that minimises
the administration and duplication of resources at the expense of some added complexity,
see figure below.
| Proposed PRISM architecture |
|
| Figure Summary This figure shows the PRISM user interacting with the central and local PRISM sites in a Web based scenario. |
It is proposed to gradually refine the system in two phases and into two different products
configured from the same software base:
Local system: used by developers to develop models without affecting operations.
Web based system: used by administraters and users to administrate model changes and
run coupled experiments remotely.
This document is primarily about delivering the infrastructure necessary to enable a distributed configuration,submission and monitoring system for PRISM experiments. A local system will be delivered to enable model developers to run the configuration tool without the web service infrastructure to test out their models. The local system is based on the same software to ensure a transparent transition between the local and the Web Services infrastructure system. The requirements and constraints from the REDOC document will be translated into a specific design in the document. The architecture will be detailed into software components and the choice of these components discussed. A run-time and deployment architecture will be discussed and finally an implementation model detailing the order and importance of implementation tasks is presented.
Software component: software that is a part of a larger system,
or an object that is part of a larger object, delivering a specific functionality.
Deployment: describes the physical distribution of the system being built, in terms of how
functionality, that is software components, are distributed among a set of run-time processing nodes
(computers). The terms is also used in a more limited scope in other documents referring to
the deployment of an experiment to computing resources (REDOC II.2).
Configuration:The arrangement of a computer system or component as defined by the number,
nature, and interconnections of its constituent parts.Also used in relation to experiments
where it defines the collection of programs, documents and data that an experiment consists of.
Figure 1 shows different aspects of the configuration process between the PRISM user and the PRISM site.
| Configuration process in PRISM |
|
| Figure 1. This figure shows the different views of the configuration process that are detailed in the document relating to the User and the Prism site. |
A large amount of effort in the PRISM project will go into modularisation of models and
standardisation of scientific input and output. Likewise, the task of deployment of models
on the web infrastructure and also the task of making it a painless exercise for a user
or developer to download a model locally and make it work needs to be given the same
attention and priority.
To succeed in the deployment of models we have to standardise procedures,i.e.
the standardisation of model builds, model runs and model configuration is critical
in view of long term maintainence and usability of the system.
A file structure must be specified that enables a locally configured model to be put into an archive file
and transported to a deployment site and deployed there automatically.
Such a structure needs to be proposed by the model developers but could look like this:
/prism
/models/modelX/src
/lib
/doc
/config
/runscripts
/bin
/tmp
A version of the GUI (Graphical User Interface) that runs without the web infrastructure
will be developed initially. This version can be started from the command line on the users
linux workstation and will not need any network support. Support for other platform is optional.
All input and output will be read/written to the current directory according to the standard
structure described above.
The user will have to download the latest versions from a central source code repository.
A Web Services (WS) system makes the services it provides abstract, shielding the client from
the particulars of an implementation of a service. A client, be it another service
or a user interface, requests a specific service.
The service is located with the help of a web services registry using
UDDI, the Universal Description, Discovery and Integration protocol, which contains information on
where the service is located and how to use it as shown in Figure 2.
| Service lookup in WS |
|
| Figure 2 This figure shows how a client requests a service from a server by looking up the service provider in a directory. |
A wide area network (WAN) is a geographically dispersed telecommunications network.
The term distinguishes a broader telecommunication structure from a local area network (LAN).
A wide area network may be privately owned or rented, but the term usually indicates the
inclusion of public (shared user) networks.
A local area network (LAN) is a group of computers and associated devices that share a
common communications line and typically share the resources of a single processor or
server within a small geographic area (for example, within an office building).
A local area network may serve as few as two or three users (for example, in a home network)
or many as thousands of users.
The client is typically running in his LAN requesting a service from applications
running in another LAN using a WAN such as the Internet and the related protocols to
forward the request.
The WS universe consist of a number of zones or locations derived from their networking status
as LAN or WAN .
The importance of the zones lies in that as messages travel from a LAN to a WAN
the authentication and validity of the message is lost because the public network
potentially allows tampering with the message. Measures need to be taken to detect
and prevent such tampering.
Below are the software components in the architecture of the
subsystems infrastructure for a generalised WS system. The
application layer describes the software component(s) in the
subsystem and its function. The middle-ware layer details the
actual implementation of the subsystem as shown in Figure 4.
| WS subsystem composition |
|
| Figure 4 This figure shows the layers that make up the subsystems in a WS architecture. |
| Table of software components in the system | |||
| Application layer | Applicatiton function | Middle-ware layer | Communications protocol |
| User Interface | Interface task | Java.Applet | HTTP |
| Web server | Serve pages,Listens for requests | Apache | HTTP |
| Application server | Interface application, | Tomcat | AJP12 |
| Domain Applications | Deliver domain function such as configurations | Any | Any |
| Authentication server | Authenticate clients | SKey, Kerberos | HTTP/HTTPS |
| ORB (Object Request Broker) | interface to application server | Java | HTTP |
| Event service | Deliver events to applications | Java | HTTP |
| Directory server | Publish and access information on services | UDDI server Java | WSDL |
| PRISM administrator | Executes administration tasks | Definitions of administration entities | SSH |
| Table 3 | |||
The processes described in the REDOC document will be translated into software that enables the desired activities.
The processes listed in the REDOC II.1 document, Tables 1,2,3,referred to as RDT1, RDT2 and RDT3 describe
what activities take place and which process is handling it. For
each process the software components that handle the related
activity is described.
The physical location of the software component also determines the choice of component,
but the mapping of components to the architecture will be more closely examined in
the deployment section .
The design of the PRISM system processes falls into
three categories mirrored by the REDOC Tables RDT1/2/3 as shown in Figure 5.
| System processes in PRISM |
|
| Figure 5 This shows the system processes categorised into controller, controlling and controlled. |
The subsystems communicates by sending messages to each other. The
messages are usually of the form request - reply, but can also be in the
form of publishing information, eg. event notifications.
In this case no reply is expected.
The term reply should be
understood as receiving a useful object back, not a protocol
confirmation of the transmission status.
Figure 13 details the messages and protocols in a WS system. The MX numbers
in Figure 13 relate to the description of the messages in Table 6.
| Table of system messages | ||
| Message | Protocol | Function |
| M1 | HTTP/S | Load gui from central webserver |
| M2 | HTTP/S | Authenticate from gui to webserver |
| M3 | AJP12/13 | All traffic to tomcat |
| M4 | RMI | Reading/writing experiment configurations, submit |
| M5 | FORK | Send job to SMS scheduler |
| M6 | QSUB | Queue job to supercomputer |
| M7 | FCGI | Monitoring requests to SMS on job status |
| M8 | RPC | Cgi server requesting information from SMS |
| M9 | AJP/RMI | M3/M7 traffic if PRISM site has no WEBSERVER |
| M10 | RMI,SOAP,RPC | M4 traffic if PRISM site has no WEBSERVER OR TOMCAT |
| M11 | FTP | Ftp of new model code to build.Requested by Prism site |
| Table 6 | ||
| Case 1: User downloads GUI and directory info. |
|
| Figure 7 This figure shows a swimlane diagram of the initialisation of the users home folders showing the transition between software components. |
| Case 1 | |||
| Message | Function | Files or information needed | |
| M1 | Load gui from central webserver | Java classes | |
| M2 | Authenticate from gui to webserver | Password information | |
| M3 | Request application | Directory information on applications available | |
| M4 | Reading users home directory | Directory listing from filesystem | |
| Table 8 | |||
| Case 2: User submits experiment. |
|
| Figure 9 This figure shows a swimlane diagram of the submission of an experiment to the remote site showing the transition between software components. |
| Case 2 | |||
| Message | Function | Files or information needed | |
| M1,2 | Submit composition to central webserver | Values of all changed variables | |
| M3 | Request application server on specific site | Directory information on applications on all PRISM sites available | |
| M4 | Generate parametrisised standard deployment scripts and SMS control scripts. | Standard scripts and all variables in composition. | |
| M5 | Submit standard deployment scripts and SMS control scripts to SMS | Standard scripts and all variables in composition. | |
| M6 | Enqueue standard deployment scripts to supercomputer | Standard scripts and all variables in composition. | |
| Table 10 | |||
| Case 3: User monitors experiment. |
|
| Figure 11 This figure shows a swimlane diagram of a request for monitoring and control information showing the transition between software components. |
| Case 3 | |||
| Message | Function | Files or information needed | |
| M1,M2 | Request job information from central webserver | Job Id | |
| M7 | Direct request to application | Directory information on applications available | |
| M8 | Request information about jobs | Job and user id | |
| Table 12 | |||
| PRISM system with messages and components |
|
| Figure 13 The figure shows a central site and a PRISM site with all components in place. |
The use of existing components is necessary to achieve the system as described in REDOC in the time frame of the project. Below is a table of the software components and their possible implementation and the state of the implementations is as follows:
| Table of software components and implementations | ||||
| Id | Application | Application layer | Implementation | State |
| 1 | Experiment configuration | User interface | PrepIFS | Exists/to be refined |
| 2 | Experiment visualisation | ?? | ?? | As described by REDOC II.3 |
| 3 | Authentication | ** | ?? | To be developed |
| 4 | Archive query | ?? | ?? | As described by REDOC II.3 |
| 5 | Documentation | User interface | PrepIFS | Exists/To be refined |
| 6 | Experiment monitoring | User interface | XCDP | Exists/To be refined |
| 7 | Experiment configuration server | Application | Java RMI | Exists/To be refined |
| 8 | Documentation server | Web Server | Apache | Exists/To be refined |
| 9 | Administration server | Application | Java | to be developed |
| 10 | Archive Meta Data browser and server | ?? | ?? | As described by REDOC II.3 |
| 11 | Visualisation server | Application | ?? | As described b REDOC II.3 |
| 12 | Monitoring server | Application | SMS | To be refined/developed |
| 13 | Scheduling server | Application | SMS | Existing |
| 14 | Execution server | Application | Coupler,Model | Existing, to be refined/developed |
| 15 | Archiving server | Application | ?? | As described by REDOC II.3 |
| 16 | Authentication server | Application | ** | To be decided |
| 17 | ORB | Application | PrepIFS | To be refined |
| 18 | Request Listener | Application | Java | To be developed |
| 19 | Event service | Application | Java Jini | To be developed |
| 20 | Directory server | Application | Java UDDI | To be developed |
| 21 | Experiment configuration | Definitions Input data | PrepIFS | To be developed |
| 22 | Administration scripts | Application | Any | To be developed |
| 23 | Semiautomatic deployment of java components | Application | Java | To be developed |
| Table 14 Legend. [Exists] : Can be without modification. [To be refined] : The component exists with the correct functionality but will need some modification. [To be developed] : The component does not exist. [**] : To be decided. See Security implementation . [??] : Has yet to be published. | ||||
PrepIFS as an experiment configuration application consists of several components: GUI,ORB and configuration server.
The GUI allows access to experiment configurations, SMIOC and SSC as described in ARCDI 2.2 - PRISM coupler and I/O library,
stored on the configuration server and also contains logic to validate the configuration.
Experiment configurations are sent to the configuration server which
generates the correct job control scripts for the experiments and deploys these through SMS which
schedules the experiment for execution.
The choice of Apache is based on the fact that it is free and used by a majority of the worlds web servers as well as being reliable, mature, maintained and high performance.
The Tomcat application server is closely linked to the Apache Web server and follows the choice of Apache and Java as implementation language of preference.(see REDOC II.1) For sites that do not provide a web server or application server the overhead to install these components is too large. Therefore a service component should be developed that does not require the web server and application server. This service component, we refer to it as a Request Listener (RL), listens for HTTP requests from the Internet and converts them into calls to the provider services listed in Table RDT2. Great care needs to be taken to prove that the RL can only forward authenticated requests and that only the intended services can be reached.
The choice of SMS/XCDP as scheduling/monitoring tool is based on our evaluation of its reliability,
functionality, performance and suitability for working with very large numbers of tasks and their dependencies.
Figure 16 shows how information is exchanged between the components.
| Monitoring and scheduling of experiments |
|
| Figure 16 This figure shows how the user can monitor experiments using the XCDP and SMS tools. |
Combining the PrepIFS system with the SMS/XCDP tools results in a configurable and flexible WS system (Fig. 5) with much of the required functionality in place as described in REDOC II.1.
The requirements for components listed in Table 14 as to be refined or to be developed
are examined further.
Experiment configuration , task id# 1
Addition of keyboard navigation.
Creation of a experiment configuration definition interface to allow modellers to develop configurations
for the models remotely (optional).
Changes to allow a more flexible experiment hierarchy.
Standardisation of interfaces.
It is assumed that this task will only involve implementation of an interface to start the visualisation tools. If there are no requirements the user interface will spawn a browser with a configurable URL.
This task contains the development of an authentication system as well as exporting the authentication as an interface to other services if needed.
It is assumed that this task will only involve implementation of an interface to start the archive tools. If there are no requirements the user interface will spawn a browser with a configurable URL.
PrepIFS has an integrated help/documentation system available. It is desirable that this is extended to allow modellers to add/change documentation remotely in a safe way.
The current monitoring system (see fig 1 in REDOC II.1) is designed for LAN access and is to be extended to WAN access over the Internet.It is desirable that it retains its graphical view if possible. It should provide the functionality as described in section II.4, if this can be achieved without security concerns.
Creation of a deployment environment and the scripts needed to deploy the composed model automatically.
Automatic generation of documentation from all available meta data and addition of developer descriptions.
An interface for administration of the WS system and domain software should be developed. Initially this will be handled manually but some automatic system is desirable. This is further developed in section Software life cycle management .
It is assumed that this task will only involve implementation of an interface to start the archive tools. If there are no requirements the user interface will spawn a browser with a configurable URL.
It is assumed that this task will only involve implementation of an interface to start the archive tools. If there are no requirements the user interface will spawn a browser with a configurable URL.
Some refinements needed to allow for web based monitoring.
No changes anticipated.
This is the task of the modelers.
It is assumed that this task will only involve implementation of an interface to start the archive tools. Due to the lack of a central archiving system archiving will be made at the site where the experiment runs using the local facilties available. This is detailed in REDOC II.3. If there are no requirements the user interface will spawn a browser with a configurable URL.
This will be implemented as recommended in collaboration with Fujitsu. It should allow for Message Authentication (MAC) between services and for client authentication between users and services.
The existing ORB has a protocol manager for Java RMI services. This will be extended to allow
other protocols such as Java-RPC. A typical system is shown in Figure 17:
| PRISM subsystem software components for communication |
|
| Figure 17 This figure shows the software components necessary to interface a client with a server using a specific protocol. |
This service component, the Request Listener (RL), listens for HTTP requests from the Internet and converts them into calls to the provider services. It allows for deployment of models from the central site to the PRISM sites. There are considerable security implications for this component and by delaying the development of this component to a later stage software to help with this task should have become available.
An event service should be developed to allow for notification between applications and to clients. A Java application programming interface JINI exists and this will be implemented.
The PrepIFS directory server, the interface repository, is a proprietary technology and this should be migrated to a UDDI compliant server.
The administration model proposed requires considerable standardisation and and automation. The process is described in section Software life cycle management.
The use of XML in definitions of meta data requires the configuration server to be able to read its input data in this format.It might be needed to create special configurations files for this purpose.
An interface for administration of the WS system and domain software should be developed. Initially this will be handled manually but some automatic system is desirable. This is further developed in section Software life cycle management .
The configuration of all system components on a central site on dedicated hardware and with security measures in place.
The web services concept is a very flexible architecture and well suited
for deployment over multiple computers.The proposed architecture involves
the partitioning of functionality into a central system for storing experiment configurations and
local PRISM sites for executing the models.
The components will be located as follows,where 1 means on site and 0 means not on site:
| Table of software components and their location | |||
| Component | Central site | PRISM site | |
| Experiment configuration server | 1 | 0 | |
| Documentation server | 1 | 0 | |
| Administration server | 1 | 0 | |
| Meta Data server | 1 | 0 | |
| Visualisation server | 0 | 1 | |
| Monitoring server | 0 | 1 | |
| Scheduling server | 0 | 1 | |
| Execution server | 0 | 1 | |
| Archiving server | 0 | 1 | |
| Authentication server | 1 | 0 | |
| ORB | 1 | 0 | |
| Event service | 1 | 1 | |
| Directory server | 1 | 0 | |
| Table 18 | |||
Large parts of the communication will take place over public networks (Internet) of which we have no control. It must be understood that communications over the Internet is on a best effort basis. An organisation (email's etc) for reporting faults in web services together with problem report standards must be set up. A monitoring system for the web services together with some diagnostic tools would be desirable.
This is connected to the requirements of the subsystems and what context related
information of the main GUI they need, for example "current experiment".
Close cooperation is needed with WP3i, Assembling the PRISM System, will be required to make sure all
configuration information can be accessed from the GUI and to provide for the scripts used to run the models.
These scripts typically deals with the:
Initially manual administration will have to be considered. Typically and administrator
has to log into the site. Tools such as Secure Shell should make this possible.
Semiautomatic administration should be developed allowing for administrators to install new
versions of models and for infrastructure components to be updated efficiently.
Component models should be updated through the administration interface.
This interface is the composition GUI used for experiments but will now enable the
administrator to enter locations for files, change compilation options and the like.
These responsible components are number 1, 9 and 22 in Table 14.
A typical administration scenario is described:
The area of access control has not been addressed by any other document and
as such is undefined for archiving, visualisation, monitoring and running
experiments.
This questions is related to the security implementation.
It is expected that unix permissions related to user ids will be
implemented and that only preregistered per user access to PRISM sites
will be allowed.
In general the implementation should be feature based,i.e. we first deliver a very basic but useful functionality. Subsequent development is concerned with developing new functionality as "features" with each new feature requiring a new release.The implementation is to be made as a local system and a WS system.
The local system will run the server and client parts as one component on a local computer. The software will be made available from a repository at prism.enes.org. The software will deliver a GUI run from the command line that reads all configuration files from the current working directory. The user will be in full control of all configuration files and can edit them at will. This system is aimed primarely at model developers, refer to local system.
The WS system involves splitting the local system into its separate software components and moving the configuration files developed with the local system to the server. This requires the security mechanisms to be in place.
The security requirements have been described in detail in the REDOC II.1 document. Some implementation aspects are :
For authentication between humans and computers we should use a software independent simple password related
system such as s/key. This system can be implemented with HTML and Java based
solutions and is well known to system administrators who already maintain password
files. Implementations are freely available. Communication can be secured by signatures but should not
be encrypted.
For authentication between computers standardisation is currently under way by the Sun Web Messages
Security API (http://jcp.org/jsr/detail/183.jsp) and also by Worl Wide Web Consortium Encryption WG (http://www.w3c.org/Encryption/2001/)
and other related XML technologies such as SAML (http://www.oasis-open.org/committees/security/).
The basics of the these technologies is to use signatures to secure the messages from being tampered with
and public key encryption to allow for authentication. An implementation building on the password files
as a shared secret key is possible and could be implemented if no standards have emerged.
The implemenentation plan builds on creating the necessary infrastructure for the WS system
on the central site . Parallel to this the local system is used by developers to work on
their models so that they can be made available to the central system.
Once the quality and operation of the system is satisfactory
the remote servers on other sites will be developed and deployed
letting users submit experiments on other sites.
This has the advantage of not spending administration efforts on
inferior quality software and time on remote debugging sessions.
The implementation plan consists of 4 different phases:
Foundations, prototype, network and integration, documentation and features.
The plan tries to move the features not necessary for basic operation to the
later phases.
It is important to determine the critical path in the development and
the functionality that can stop the progress of the project:
It is essential to determine the critical path for the implementation of components. Dependencies are given for each phase.
The fact that other projects with similar goals (Globus etc) are emerging and being developed raises the question of integration and also cooperation. If a high degree of inter-operability is required we should test their systems and also make contact to discuss how we can both benefit from each others development. Advantages could be reuse of software but the time it takes to identify these areas is at stake.
| Table of risks | |||
| Risk | Risk Magnitude | Description | Impact |
| Security demands incompatible on some sites. | Severe | Multiple security solutions may be necessary. | If sites cannot agree on one security solution it may introduce costly separate solution affecting the client experience. |
| Deployment and administration cycles made difficult due to security. | Severe | Increased manual intervention necessary at higher cost. | Synchronisation of software is delayed. |
| Hardware resources unavailable for testing and implementing the infrastructure. | Fatal | System cannot be built and tested. | No system. |
| Standardisation of build scripts not achieved. | Severe | Models not conforming to standards. | Manual intervention required. |
| Creation of configuration information for GUI incomplete. | Severe | Necessary information missing for configuration or deployment | GUI will not work properly and automation benefits lost. |
| Deployment standards not achieved. | Severe | Difficult to achieve standards if validation and editing tools not available. | Deployment will fail. |
| Table 19 | |||
| Table of Definitions,Acronyms and Abbreviations | |||
| Keyword | Description | ||
| WSDL. | is an XML format for describing network services as a set of endpoints operating on messages containing either document-oriented or procedure-oriented information. | ||
| XML | The Extensible Markup Language is the universal format for structured documents and data on the Web. is a human-readable, machine-understandable, general syntax for describing hierarchical data, applicable to a wide range of applications Custom tags enable the definition, transmission, validation, and interpretation of data between applications and between organisations. | ||
| Simple Object Access Protocol | SOAP is a lightweight protocol for exchange of information in a decentralised, distributed environment. It is an XML based protocol. | ||
| Java | A programming language invented by Sun which runs on any platform and supports web services. | ||
| SSL | Secure Socket Layer, a Netscape invented secure communications protocol. | ||
| SSH | Secure Shell, a secure remote access protocol pioneered by BSD enabling remote logins. | ||
| Kerberos | Network security system developed by MIT. | ||
| SKey | One-Time Password system. | ||
| UDDI | Universal Description, Discovery and Integration.Enables dynamic lookup and advertising of services. | ||
| X509 | Standard for certificates used by SSL authentication and encryption. | ||
| Table 20 | |||