The Oyster In a Matrix
Even if a big event happens – a major earthquake or the huge number of people reporting on the recent elections in Iran – the system we developed never breaks down. It makes full use of the Grid, with resources distributed all over the place and clever algorithms designed to fetch things when they’re needed.
The design of a generic middleware for efficient search and retrieval of annotated content in a distributed Grid environment has been motivated by the requirements posed by the European Research project. GREDIA’s main objective is the development of a Grid application platform, providing highlevel support for the implementation of Grid business applications through a flexible graphical user interface. This generic platform will facilitate the provision of business services, which mainly demand access and sharing of large quantities of distributed annotated numerical and multimedia content. Furthermore, the GREDIA platform will exploit Grid technologies to enable access to the distributed content by mobile devices. The establishment of the described functionality faces great challenges regarding data management, security, authentication and authorization mechanisms. In the heart of such a platform lies the middleware subsystem that essentially performs all data management operations for the system.
Scientists have embraced the Grid, but businesses have held back, concerned about complexity and security . A European research team has built a platform opening the Grid’s vast resources to business users. Three years ago, GREDIA {Grid enabled access to rich mEDIA content} set out to create and test a toolkit that would help businesses exploit the enormous data storage and computational power of the Grid.
“Many business applications need to work fast and need to work with huge amounts of data,” says Nikos Sarris, GREDIA’s coordinator. “The Grid is ideal for that, but software developers don’t use it because they don’t know how.”
The platform that GREDIA has now built, Sarris says, will help business application developers exploit the Grid without needing to become experts on Grid technology. Grids can flexibly link any number of computers and other devices. They offer many advantages, including almost limitless data storage and computational power plus great flexibility and reliability.
“The system we developed is extremely reliable because it’s distributed among many machines,” says Sarris. “And it optimises business transactions through clever algorithms that make the most of the grid’s distributed resources.”
{The work done here is part of the GREDIA Project – GREDIA [FP6 34363 - Grid enabled access to rich media content] which will address this problem with the provision of a Grid application development platform.}
Grid-powered journalism
The team tested their system by developing and demonstrating two business services. The first showcased news gathering and dissemination. News organisations today need to be quick and flexible. In addition to staff reporters, many use freelancers and even eyewitnesses who may have photos or video of an event, or who can use their mobile phones to describe what’s going on as it happens. GREDIA’s software lets any number of sources using almost any kind of device be turned into a news gathering team. The group’s key partner, SYMBIAN, made improvements in their operating system for mobile services to let mobile devices link into a grid.
Knowing how important security would be for business users, GREDIA built it into the development process. Program developers specify the roles different team members will play and define the rules each must follow, including rules for identification, authentication and encryption.
“Basically, what you send can not be seen by others until both the sender and receiver have been identified and authenticated,” says Sarris.
These tools let a news agency turn an ad-hoc group of contributors, editors and producers into a functional and secure virtual organisation. Running in the background, the software manages all the steps involved in recruiting and authenticating contributions, gathering and storing the information that pours in, and producing news items.
{The SIMDAT project takes industrial product and process development forward by introducing advanced Grid technology in the automotive, aerospace, pharmaceutical and meteorological verticals. – The solution of industrially relevant complex problems using data-centric Grid technology}
The Grid is ideal for this kind of application because it can handle events of any magnitude. Sarris also sees a future for GREDIA in the growing area of social media, which many people tune into almost constantly not just to keep in touch with others but also to track events they are interested in on a minute-by-minute basis. “Real-time microblogging services like Twitter produce terabytes of data, You can’t cope with that without clever ways of distributing and managing your data.” he added.
Crunching credit on the Grid
To further showcase the flexibility of the platform and middleware, the team developed and tested a grid-based application from a very different enterprise – banking. They tackled the interaction between potential lenders and borrowers. Lenders can use their home computers or even handheld devices to provide information, such as balance sheets and credit history, securely. The program authenticates the information, combines it into a profile, and calculates credit rankings using a protocol specified by the lender.
“For one person this isn’t very demanding,… But if you think of how many customers and banks there are, and realise that banks need to calculate the creditability of their customers on a regular basis, the scale changes.” Sarris says.
GREDIA’s Objectives include…
- To develop a reliable Grid aplication development platform with high-level support for the design, implementation and operational deployment of secure Grid business applications.
- To maintain a novel generic Grid middleware required for servicing business applications accessing distributed rich media content.
- To support mobile services being built to allow mobile devices to participate in the Grid Virtual Organisation in a seamless way.
- To ensure the protection of data and transaction at all levels through a dedicated security framework.
- To validate the platform by producing two pilot Grid applications to address real life scenarios servicing the domains of media and banking.
GREDIA successfully tested this credit-ranking programme on the customer base of a single bank. However, since the entire process uses the grid, it can readily be scaled up to almost any size. This makes it really easy for a bank to have reliable and fast credit checks of all its customers, but it could easily scale up to include any number of banks and individuals, or even to serve as a central authority for calculating creditability in real time.
GREDIA will provide amongst others…
- Innovative search and retrieval mechanisms for fast access of annotated numerical and multi media content distributed over the Grid, based on P2P overlay networks technology.
- A new notation for Grid application developers allowing for easy defi nition of Grid application’s static and dynamic aspects, hiding of the complexity of semantic service matchmaking.
- Ontological services supporting interoperability in and across Virtual Organisations, building a semantic abstraction layer over available resources in the Grid.
- Implementation of information services that rapidly catalogue content on mobile devices and design of Agents that seek data caching and replication services based on QoS criteria.
- Protection of data and transactions through a secure framework for authentication of entities, confidentiality and integrity to accesses from remote resources.
Sarris is eager for businesses and business application developers to use what the GREDIA team has built, all of which is available in open source along with instructions and examples. “We’ve tried to provide the technology to ‘gridify’ business applications,” he says. “It’s there, it’s available, and it’s free, so developers can just grid it, whatever it is.”
