Automatic configuration for cloud workloads

With the increase in the adoption of Cloud computing and big data processing systems, a common way to deploy analytics workloads is to acquire on-demand resources in a Cloud environment whenever the workloads need to execute. Public Cloud providers offer numerous infrastructure choices to Cloud users. For example, the number of different Virtual Machine (VM) instances offered by the Cloud providers has been steadily increasing, now numbering in dozens. This enables Cloud users to have access to the type of instance that fits their use case. However, this blessing can also be a curse since selecting the right resources is often not straightforward. When executing analytics workloads in the Cloud, ensuring the allocation of the right resources is paramount to achieve cost efficiency and satisfy strict service-level objectives (SLOs) on job completion times. This requires users to decide the number of instances (VMs) and type of instances for their deployments (together forming a Cloud configuration). In addition, the software platforms for big data processing have system parameters that need to be appropriately set. The choice of these cloud configurations and system parameter values dictate the performance and cost of the whole deployment. This thesis aims to reduce the decision burden on the users when deploying workloads in the Cloud. Our primary focus is to allow users to concentrate on defining the performance objectives rather than determining the resource allocation for their Cloud workloads. Specifically, we investigate ways to automatically determine configurations for Cloud workloads given user-specified performance and cost objectives. These workloads include jobs running on batch and stream processing systems and serverless functions. The configurations under consideration in this thesis include cloud configurations and system parameters. More concretely, in this thesis, we discuss automatic Cloud configuration optimization in two contexts. First, for distributed batch data processing systems, a Cloud ...