Diplo Cloud: Efficient and Scalable Management of RDF Data in the Cloud

0
506
Diplo Cloud: Efficient and Scalable Management of RDF Data in the Cloud

Diplo Cloud: Efficient and Scalable Management of RDF Data in the Cloud

Abstract

Diplo Cloud: Efficient and Scalable Management of RDF Data in the Cloud,Despite recent advances in distributed RDF data management, processing large-amounts of RDF data in the cloud is still very challenging. In spite of its seemingly simple data model, RDF actually encodes rich and complex graphs mixing both instance and schema-level data. Sharding such data using classical techniques or partitioning the graph using traditional min-cut algorithms leads to very inefficient distributed operations and to a high number of joins. In this paper, we describe DiploCloud, an efficient and scalable distributed RDF data management system for the cloud. Contrary to previous approaches, DiploCloud runs a physiological analysis of both instance and schema information prior to partitioning the data. In this paper, we describe the architecture of DiploCloud, its main data structures, as well as the new algorithms we use to partition and distribute data. We also present an extensive evaluation of DiploCloud showing that our system is often two orders of magnitude faster than state-of-the-art systems on standard workloads.

System Configuration

H/W System Configuration
Speed                   : 1.1 GHz
RAM                      : 256 MB(min)
Hard Disk              : 20 GB
Floppy Drive          : 1.44 MB
Key Board             : Standard Windows Keyboard
Mouse                  : Two or Three Button Mouse
Monitor                : SVGA
S/W System Configuration

Platform                     :  cloud computing

Operating system       : Windows Xp,7,
Server                       : WAMP/Apache
Working on                : Browser Like Firefox, IE

Conclusion

DiploCloud is an efficient and scalable system for managing RDF data in the cloud. From our perspective, it strikes an optimal balance between intra-operator parallelism and data co-location by considering recurring, fine-grained physiological RDF partitions and distributed data allocation schemes, leading however to potentially bigger data (redundancy introduced by higher scopes or adaptive molecules) and to more complex inserts and updates. DiploCloud is particularly suited to clusters of commodity machines and cloud environments where network latencies can be high, since it systematically tries to avoid all complex and distributed operations for query execution. Our experimental evaluation showed that it very favorably compares to state-of-the-art systems in such environments. We plan to continue developing DiploCloud in several directions: First, we plan to include some further compression mechanisms (e.g., HDT ). We plan to work on an automatic templates discovery based on frequent patterns and untyped elements. Also, we plan to work on integrating an inference engine into DiploCloud to support a larger set of semantic constraints and queriesnatively.Finally,wearecurrently testing and extending our system with several partners in order to manage extremely-large scale, distributed RDF datasets in the context of bioinformatics applications.