A distributed publish-subscribe architecture for XML-based event dissemination

Abstract

The aim of this study is to investigate how a pub-sub architecture can be efficiently empowered with XML-based filtering capability in a distributed environment. An XML based pub-sub system mainly focus on active, selective, asynchronous dissemination of timely, personalized and dynamic information represented and modelled by XML. It supports improved filtering flexibility and expressiveness as compared to its predecessor, the topic-based and content-based pub-sub systems. In recent years, XML based pub-sub shows increasing significance as the evolution of web 2.0 technologies and the succeeding proliferation of social network services demand a better information dissemination paradigm that is suitable for information-driven type of applications. However, many existing approaches in XML-based pub-sub do not operate efficiently and do not scale well in a wide-area environment, such as the Internet. In addition, most of the studies require dedicated infrastructure support and do not concern about fault-tolerance of the system. In this thesis, we introduced a series of novel approaches that contribute toward enabling an efficient distributed XML-based pub-sub system design that scale up to operate in a wide-area and large-scale environment. Our design possesses self-organizing capability for the overlay infrastructure and supports fault-tolerance that matches the dynamism of the underlying network. In our design, we combine the process of data filtering with routing where we exploit the structure coverage relation between XML representation of subscriptions and publications. Here, the subscriptions are forwarded selectively to only a few rendezvous nodes without the need to flood subscriptions across the entire network. At the rendezvous node, the incoming publications can be evaluated against subscriptions and delivered to subscribers if matched. To achieve this, we applied a bloom filter-based filtering approach that coalesced into the addressing scheme for the Key-based Routing to provide a scalable, flexible and robust pub-sub infrastructure. In addition, we developed a hypercube overlay as a multicasting infrastructure for efficient dissemination of publications. We further extended proposed architecture by developing a redundancy-based fault-tolerance strategy to enhance the robustness of the system considering the dynamism of the underlying network environment. We validate the efficiency and scalability of the proposed system through extensive experiments. It shows that the proposed system is able to balance the load among the peers and to prune out unmatched publication messages at the early stages along the dissemination path. It also shows that the proposed system scales well with increasing number of peers, subscriptions and publication events. We also evaluated the availability and effectiveness of the fault-tolerant capability in the case of nodes churning. The evaluation result shows that proposed approach can work very well in a dynamic network environment where nodes may join, leave and fail at times.