International Journal of Big Data Management (IJBDM), (in press), Inderscience

Most techniques for data processing run on standard infrastructure management systems, while large data sets being increasingly generated. The main challenge refers to using technology to gather efficient and faster insights from a dataset, having in mind not to ask what data is easily obtainable and which tools are most easily amenable to working with that dataset, but rather what question the analysis is trying to answer. This creates a landscape with data-intensive projects that prioritize technical prowess of execution over the robustness of analytical findings. Hence, a data-driven stack for Big Data applications’ management and deployment is being described, namely Diastema, bringing efficient data management through distributed storage and analytics approaches, aiming at high performance and utilization of heterogeneous resources, including abstraction, gateways, and small-footprint Virtual Machines. Through a set of innovative mechanisms, Diastema will turn limited-value raw data to timely, relevant data to be exploited by businesses.