Spark is a more efficient distributed big data processing framework following Hadoop. It provides users with more than 180 adjustable configuration parameters, and how to choose the optimal configuration automatically to make the Spark application run effectively is challenging. The key to address the above challenge is having the ability to predict the performance of Spark applications in different

With the rise of concerns over security and privacy in the cloud, the “security-on-demand” service mode dynamically provides cloud customers with trusted computing environments according to their specific security needs. Major challenges, however, remain to achieve this goal: (1) integrating an auditable, tamper-resistant trust-management mechanism into the cloud infrastructure and (2) building a protocol

A fundamental layer of smart cities, the Internet of Vehicles (IoV) can significantly improve transportation efficiency, reduce energy consumption, and traffic accidents. However, because of the vehicle and the RoadSide Units (RSU) use wireless channels for communication, the risk of information being leaked or tampered is highly increased. Therefore, secure and reliable authentication and key agreement

As a distributed hash table (DHT) routing overlay, Skip Graph is used in a variety of peer-to-peer (P2P) systems including cloud storage. The overlay connectivity of P2P systems is negatively affected by the arrivals and departures of nodes to and from the system that is known as churn. Preserving connectivity of the overlay network (i.e., the reachability of every pair of nodes) under churn without

To attract more customers, a cloud provider tends to give some discounts to a customer if he/she rents a plenty of resources. Under this situation, a group of customers who need homogeneous cloud instances with various deadlines are prone to purchasing resources in a collaborative manner, i.e., using a coalition game, to reduce purchase costs. It is essential to design a mechanism that enables all

With the development of blockchain-based digital currencies, the security of digital wallets becomes more and more important. As far as we know, there is no safe lightweight wallet in hyperledger fabric. To solve the problem, we proposed a Trustzone-based Secure Lightweight Wallet for Hyperledger Fabric (hereafter referred to as TSLWHF). Firstly, we designed an Unspent Transaction Output (UTXO) set

Quality of service (QoS) provision has become an important aspect of high-performance computing interconnection networks. Proof of that is the inclusion of mechanisms targeted to the provision of QoS by the main interconnection technologies such as Gigabit Ethernet, Infiniband (IB) and Omni-Path (OPA). A key component of QoS provision is the output scheduling algorithm, which determines when a packet

Graph-tensor learning operations extend tensor operations by taking the graph structure into account, which have been applied to diverse domains such as image processing and machine learning. However, the running time of graph-tensor operations increases rapidly with the number of nodes and the dimension of data on nodes, making them impractical for real-time applications. In this paper, we propose

K Nearest Neighbor (KNN) joins are used in scientific domains for data analysis, and are building blocks of several well-known algorithms. KNN-joins find the KNN of all points in a dataset. This paper focuses on a hybrid CPU/GPU approach for low-dimensional KNN-joins, where the GPU may not yield substantial performance gains over parallel CPU algorithms. We utilize a work queue that prioritizes computing

Recent trends in high-performance computing and deep learning have led to the proliferation of studies on large-scale deep neural network training. However, the frequent communication requirements among computation nodes drastically slows the overall training speeds, which causes bottlenecks in distributed training, particularly in clusters with limited network bandwidths. To mitigate the drawbacks

Machine learning over distributed data stored by many clients has important applications in use cases where data privacy is a key concern or central data storage is not an option. Recently, federated learning was proposed to solve this problem. The assumption is that the data itself is not collected centrally. In a master–worker architecture, the workers perform machine learning over their own data

Communication in data centers often involves many parallel flows that all share the same performance goal (e.g. to minimize the average completion time). A useful abstraction, coflow, is proposed to express the communication requirements of prevalent data parallel paradigms such as MapReduce and Spark. The multiple coflow routing and scheduling problem makes it challenging to derive a good theoretical

Most existing graph frameworks for GPUs adopt a vertex-centric computing model where vertex to thread mapping is applied. When run with irregular graphs, we observe significant load imbalance within SIMD-groups using vertex to thread mapping. Uneven work distribution within SIMD-groups leads to low utilization of SIMD units and inefficient use of memory bandwidth. We introduce Graph-Waving (GW) architecture

Collaborative Deep Learning (CDL) utilizes the strong feature learning capability of neural network and the model fitting robustness to solve the problem that the performance of Recommender System drops dramatically when the data is sparse. However, it makes the model training become difficult to maintain when Recommender System faces a large amount of data, and a variety of unpredictable problems

Nearest neighbors search is a core operation found in several online multimedia services. These services have to handle very large databases, while, at the same time, they must minimize the query response times observed by users. This is specially complex because those services deal with fluctuating query workloads (rates). Consequently, they must adapt at run-time to minimize the response times as

The development of cloud-assisted electronic health system effectively addresses the drawbacks of traditional medical management system. However, some challenging problems such as security and privacy in data storage and sharing cannot be ignored. First, it is difficult to ensure the integrity of electronic health records (EHRs) during the data outsourcing process. Second, it is difficult to guarantee

In this paper, we introduce a new I/O characteristic discovery methodology for performance optimizations on object-based storage systems. Different from traditional methods that select limited access attributes or heavily reply on domain knowledge about applications’ I/O behaviors, our method enables capturing data-access features as many as possible to eliminate human bias. It utilizes a machine-learning

IoT environments are continuously changing. Changes may come from the service, connectivity, or physical layers of the IoT architecture. Therefore, to function appropriately, the system needs to dynamically adapt to its environment. In previous work, we posited eight challenges to foster adaptation through all architecture layers of IoT systems. In this paper, we address the challenges to manage the

Although some existing collaborative intrusion detection schemes can increase the detection performance by dynamically allocating detection resources in smart grids, these related works fail to consider the optimization of resource allocation between IDSs under the condition of resource restriction. In this paper, considering the effect of resource restriction, we propose a resource allocation scheme

Interactive consistency is the problem in which n distinct nodes, each having its own private value, where up to t may be Byzantine, run an algorithm that allows all non-faulty nodes to infer the values of each other node. This problem is relevant to critical applications that rely on the combination of the opinions of multiple peers to provide a service. Examples include monitoring a content source

Network-on-chip (NoC) is an integral part of many-core microprocessors. Performance analysis of network-on-chip directly affects the performance of the microprocessor. In this paper we propose a mathematical model to represent packet flow in an NoC as an open feed-forward queuing network. We study the performance of NoC by varying different parameters that includes packet injection rate, packet size

Graphics Processing Units technology (GPU) and CUDA architecture are one of the most used options to adapt machine learning techniques to the huge amounts of complex data that are currently generated. Biclustering techniques are useful for discovering local patterns in datasets. Those of them that have been implemented to use GPU resources in parallel have improved their computational performance.

Rendezvous algorithms encode a communication pattern that is useful when processors sending data do not know who the receiving processors should be, or vice versa. The idea is to define an intermediate decomposition where datums from different sending processors can ”rendezvous” to perform a computation, in a manner that both the senders and eventual receivers of the results can identify the appropriate

A fast parallel graph partitioner can benefit many applications by reducing data transfers. The online methods for partitioning graphs have to be fast and they often rely on simple one-pass streaming algorithms, while the offline methods for partitioning graphs contain more involved algorithms and the most successful methods in this category belong to the multilevel approaches. In this work, we assess

Shared-memory graph processing is usually more efficient than in a cluster in terms of cost effectiveness, ease of programming and runtime. However, the limited memory capacity of a single machine and the huge sizes of graphs restrains its applicability. Hence, it is imperative to reduce memory footprint. We observe that index compression holds promise and propose CIC-PIM, a lightweight encoding with

Graph Neural Networks (GNNs) are powerful deep learning models to generate node embeddings on graphs. When applying deep GNNs on large graphs, it is still challenging to perform training in an efficient and scalable way. We propose a novel parallel training framework. Through sampling small subgraphs as minibatches, we reduce training workload by orders of magnitude compared with state-of-the-art minibatch

The classification tree (CT) may be used to establish explicit classification rules for Satellite Imagery (SI). However, the accuracy of explicit classification rules attained by this method is poor. Back-propagation networks (BPN) and the support vector machine (SVM) may both be used to establish highly accurate models for predicting the classification of SI. However, neither is able to generate explicit

Reliability evaluation of interconnection networks is of significant importance to the design and maintenance of interconnection networks. The extra edge-connectivity and component edge-connectivity are two important parameters for the reliability evaluation of interconnection networks. In this paper, we determine the h-extra edge-connectivity of an n-dimensional augmented cube for h≤2[n2], n≥2 and

Cloud computing provides scalable network services and makes network management more flexible by combining Software-Defined Networking (SDN). Through the northbound interface (e.g., REST API) offered by the SDN controller, users can easily deploy diversified applications to access the network resources. However, exploiting the openness of the northbound interface, malicious applications abuse APIs

The upcoming digitalization in the context of Cyber-physical Systems (CPS), enabled through Internet-of-Things (IoT) infrastructures, require efficient methods for distributed processing of the data, that is generated by multiple sources. We address the problem of obstacle detection and localization through data clustering, which is a common component for data processing in the fusion of multiple point

The constructions of node-disjoint paths have been well applied to the study of connectivity, diameter, parallel routing, reliability, and fault tolerance of an interconnection network. In order to minimize the transmission cost and latency, the total length and maximal length of the node-disjoint paths should be minimized, respectively. The construction of node-disjoint paths with their maximal length

This paper presents a novel leaderless protocol (FPC-BI: Fast Probabilistic Consensus within Byzantine Infrastructures) with a low communicational complexity and which allows a set of nodes to come to a consensus on a value of a single bit. The paper makes the assumption that part of the nodes are Byzantine, and are thus controlled by an adversary who intends to either delay the consensus, or break

We design distributed algorithms to compute approximate solutions for several related graph optimization problems. All our algorithms have round complexity being logarithmic in the number of nodes of the underlying graph and in particular independent of the graph diameter. By using a primal–dual approach, we develop a 2(1+ϵ)-approximation algorithm for computing the coreness values of the nodes in

The interconnection network is a key element in High-Performance Computing (HPC) and Datacenter (DC) systems whose performance depends on several design parameters, such as the topology, the switch architecture, and the routing algorithm. Among the most common topologies in HPC systems, the Fat-Tree offers several shortest-path routes between any pair of end-nodes, which allows multi-path routing schemes

With the advent of the Industry 4.0 era, the development of smart cities based on the Internet of Things (IoT) has reached a new level. As a key component of the Internet of Things (IoT), the security of wireless sensor networks (WSN) has received widespread attention. Among them, Energy Internet, as an important part to support the construction of smart cities, its security and reliability research

Machine Learning (ML) has become a critical tool enabling new methods of analysis and driving deeper understanding of phenomena across scientific disciplines. There is a growing need for “learning systems” to support various phases in the ML lifecycle. While others have focused on supporting model development, training, and inference, few have focused on the unique challenges inherent in science, such

The Dragonfly topology is currently one of the most popular network topologies in high-performance parallel systems. The interconnection networks of many of these systems are built from components based on the InfiniBand specification. However, due to some constraints in this specification, the available versions of the InfiniBand network controller (OpenSM) do not include routing engines based on

Mobile crowdsourcing is being increasingly used by industrial and research communities to build realistic datasets. By leveraging the capabilities of mobile devices, mobile crowdsourcing apps can be used to track participants’ activity and to collect insightful reports from the environment (e.g., air quality, network quality). However, most of existing crowdsourced datasets systematically tag data

Service migration in mobile edge computing is a promising approach to improving the quality of service (QoS) for mobile users and reducing the network operational cost for service providers as well. However, these benefits are not free, coming at costs of bulk-data transfer, and likely service disruption, which could consequently increase the overall service costs. To gain the benefits of service migration

In this study, we develop a novel data reconstruction technique for parallel storage systems housed in modern data centers. We advocate for erasure-coded data storage systems to archive warm data (a.k.a., unpopular data), which attract a limited number of accesses or updates. Different from hot or cold data, warm data have to be treated in a distinctive way to optimize system performance and storage-space

The NP-hard problem of task scheduling with communication delays (P|prec,cij|Cmax) is often tackled using approximate methods, but guarantees on the quality of these heuristic solutions are hard to come by. Optimal schedules are therefore invaluable for properly evaluating these heuristics, as well as being very useful for applications in time critical systems. Optimal solving using branch-and-bound

The technology-driven shift in distributed network systems, Internet technology and energy grid digitization have changed the traditional spoke-hub distribution models to the more flexible and giant network-to-networks paradigm known as Energy Cloud. This energy cloud paradigm has the ability to adapt to the large-scale distributed energy resources with dynamic demand-response associated with smart

Large-scale data-intensive applications provide services to users by routing service requests to geographically distributed data centers interconnected by Internet links. In order to achieve good reliability and data access latency performance, cloud service providers often simultaneously place multiple copies of the data in different data centers. The network communication required for updating the

Infrastructure-as-a-service (IaaS) Clouds concurrently accommodate diverse sets of user requests, requiring an efficient strategy for storing and retrieving virtual machine images (VMIs) at a large scale. The VMI storage management requires dealing with multiple VMIs, typically in the magnitude of gigabytes, which entails VMI sprawl issues hindering the elastic resource management and provisioning

Fog computing is an emerging paradigm that brings computing capabilities closer to distributed IoT devices, which provides networking services between end devices and traditional cloud data centers. One important mission is to further reduce the monetary cost of fog resources while meeting the ever-growing demands of multiple users. In this paper, we focus on minimizing the total cost for multiple

Self-stabilizing and silent distributed algorithms for token distribution in rooted tree networks are given. Initially, each process of a graph holds at most ℓ tokens. Our goal is to distribute the tokens uniformly in the whole network so that every process holds exactly k tokens. In the initial configuration, the total number of tokens in the network may not be nk where n is the number of processes

We study leader election in unidirectional rings of homonyms that have no a priori knowledge of the number of processes. In this context, we show that there exists no algorithm that solves the process-terminating leader election problem for the class of asymmetrically labeled unidirectional rings. More precisely, we prove that there is no process-terminating leader election algorithm even for the subclass

Elephant flows (elephants) refer to the sequences of packets that contribute only 10% of the total volume but consume over 90% of the network bandwidth. They often cause network congestion and should be efficiently managed. Present cloud data centers often involve host- and switch-based approaches to detect and schedule elephants, but suffer (1) each host and switch in the network needs to be customized

3D tomographic imaging requires the computation of solutions to very large inverse problems. In many applications, iterative algorithms provide superior results, however, memory limits in available computing hardware restrict the size of problems that can be solved. For this reason, iterative methods are not normally used to reconstruct typical data sets acquired with lab based CT systems. We thus

Emerging scientific workflows running at extreme scale are composed of multiple applications that interact and exchange data at runtime. While staging-based approaches, e.g. in-situ/in-transit processing, are promising, dynamic behaviors (e.g. data volumes and distributions) in coupled applications and varying resource constraints at runtime make the efficient use of these techniques challenging. Addressing

Early-stage disease risk prediction can be beneficial to improve the health of the mass and can reduce the economic burden of late treatment. Machine learning has played a pivotal role in predictive systems, which requires achieving a specific degree of accuracy for healthcare systems. Most recently researchers have found the necessity of bridging between epidemiology and machine learning classifications

With the popularity of the Internet of Things (IoT), data integrity verification in the edge cloud storage attracts attentions from many researchers. Due to the over dependence of the Third Party Auditor (TPA) and the dynamical nature of the IoT data, the traditional data integrity verification framework for cloud storage can hardly work. To satisfy the characteristics of the IoT and avoid the over

In this paper, a new distributed multi-agent framework based on the three layers’ fog computing architecture is developed for real-time microgrid economic dispatch and monitoring. To this end, the changes of load at any time will be tracked by the proposed technique, considering unit sudden exits and entries. Moreover, to make the system more realistic, different renewable energies, including photovoltaics

Stochastic gradient descent (SGD) is a popular stochastic optimization method in machine learning. Traditional parallel SGD algorithms, e.g., SimuParallel SGD (Zinkevich, 2010), often require all nodes to have the same performance or to consume equal quantities of data. However, these requirements are difficult to satisfy when the parallel SGD algorithms run in a heterogeneous computing environment;

Machine learning and artificial intelligence (AI) applications often rely on performing many small matrix operations—in particular general matrix–matrix multiplication (GEMM). These operations are usually performed in a reduced precision, such as the 16-bit floating-point format (i.e., half precision or FP16). The GEMM operation is also very important for dense linear algebra algorithms, and half-precision

Herlihy showed that multiprocessors must support advanced atomic objects, such as compare-and-swap, to be able to solve any arbitrary synchronization task among any number of processes (Herlihy, 1991). Elementary objects such as read-write registers and fetch-and-add are fundamentally limited to at most two processes with respect to solving an arbitrary synchronization task. Later, it was also shown

Smart Cities evolve into complex and pervasive urban environments with a citizens’ mandate to meet sustainable development goals. Repositioning democratic values of citizens’ choices in these complex ecosystems has turned out to be imperative in an era of social media filter bubbles, fake news and opportunities for manipulating electoral results with such means. This paper introduces a new paradigm

Multi/Many-core systems based on the traditional electrical network-on-chip are confronted with the limited bandwidth, high latency, and reliability challenges. The emerging optical solution of silicon photonics has promised to improve the design parameters of electrical interconnections for future multiprocessor system on chips. Although the optical network-on-chip has advantages in terms of reliability

The technological evolution of the smart grids is going to take its shape in the form of a new paradigm called the Internet of Energy (IoE); which is considered to be the convergence of internet, communication, and energy. Like other evolved technologies, the IoE inherits security vulnerabilities from its constituents that need to be addressed. Intrusion Detection Systems (IDS) have been used to counteract