publications | Internet Computing and Emerging Technologies lab (ICET-lab)

2025

EASE

Emotional Strain and Frustration in LLM Interactions in Software Engineering

Cristina Martinez Montes, and Ranim Khojah

Proceedings of the ACM International Conference on Evaluation and Assessment in Software Engineering (EASE 2025), 2025

Bib

@article{montes:25,
  author = {Montes, Cristina Martinez and Khojah, Ranim},
  title = {Emotional Strain and Frustration in LLM Interactions in Software Engineering},
  journal = {Proceedings of the ACM International Conference on Evaluation and Assessment in Software Engineering (EASE 2025)},
  year = {2025},
}

2024

EAAI

A Unified Active Learning Framework for Annotating Graph Data foor Regression Tasks

Peter Samoaa, Linus Aronsson, Antonio Longa, Philipp Leitner, and Morteza Haghir Chehreghani

Journal of Engineering Applications of Artificial Intelligence, 2024

Bib PDF

@article{samoaa:24,
  author = {Samoaa, Peter and Aronsson, Linus and Longa, Antonio and Leitner, Philipp and Chehreghani, Morteza Haghir},
  title = {A Unified Active Learning Framework for Annotating Graph Data foor Regression Tasks},
  journal = {Journal of Engineering Applications of Artificial Intelligence},
  year = {2024},
}

FSE Visions 2024

The lion, the ecologist and the plankton: a classification of species in multi-bot ecosystems

Dimitrios Platis, Linda Erlenhov, and Francisco Gomes Oliveira Neto

In Proceedings of the ACM International Conference on the Foundations of Software Engineering (FSE 2024) - Ideas, Visions, and Reflections Track , Porto de Galinhas, Brazil, 2024

Bib

@inproceedings{platis:24,
  title = {{The lion, the ecologist and the plankton: a classification of species in multi-bot ecosystems}},
  booktitle = {{Proceedings of the ACM International Conference on the Foundations of Software Engineering (FSE 2024) - Ideas, Visions, and Reflections Track}},
  author = {Platis, Dimitrios and Erlenhov, Linda and de Oliveira Neto, Francisco Gomes},
  year = {2024},
  doi = {10.1145/3663529.3663782},
  pages = {482–486},
  numpages = {5},
  location = {Porto de Galinhas, Brazil},
  isbn = {9798400706585},
}

AIware @ FSE 2024

From Human-to-Human to Human-to-Bot Conversations in Software Engineering

Ranim Khojah, Francisco Gomes Oliveira Neto, and Philipp Leitner

In Proceedings of the The First International Conference on AIware (AIware) , Porto de Galinhas, Brazil, 2024

Bib PDF

@inproceedings{khojah:24:2,
  title = {{From Human-to-Human to Human-to-Bot Conversations in Software
    Engineering}},
  booktitle = {{Proceedings of the The First International Conference on AIware (AIware)}},
  author = {Khojah, Ranim and de Oliveira Neto, Francisco Gomes and Leitner, Philipp},
  year = {2024},
  isbn = {9798400706851},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  doi = {10.1145/3664646.3664761},
  pages = {38–44},
  numpages = {7},
  keywords = {Conversational agents, Dialogue characteristics, Software development},
  location = {Porto de Galinhas, Brazil},
}

FSE 2024

Beyond Code Generation: An Observational Study of ChatGPT Usage in Software Engineering Practice

Ranim Khojah, Mazen Mohamad, Philipp Leitner, and Francisco Gomes Oliveira Neto

In Proceedings of the ACM International Conference on the Foundations of Software Engineering (FSE 2024) , 2024

Bib PDF Supp

@inproceedings{khojah:24,
  title = {{Beyond Code Generation: An Observational Study of ChatGPT Usage in Software Engineering Practice}},
  booktitle = {{Proceedings of the ACM International Conference on the Foundations of Software Engineering (FSE 2024)}},
  author = {Khojah, Ranim and Mohamad, Mazen and Leitner, Philipp and de Oliveira Neto, Francisco Gomes},
  year = {2024},
  issue_date = {July 2024},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  volume = {1},
  doi = {10.1145/3660788},
}

JSS
An empirical investigation on the competences and roles of practitioners in Microservices-based Architectures

Hamdy Michael Ayas, Regina Hebig, and Philipp Leitner

Journal of Systems and Software, 2024

Abs Bib Supp

Microservices-based Architectures (MSAs) are gaining popularity since, among others, they enable rapid and independent delivery of software at scale, facilitating the delivery of business value. Additionally, there are attempts towards understanding practitioners’ roles and technical knowledge. MSAs call for affinity in several technologies as well as business domains. This diversity makes it challenging to scope and describe the roles of practitioners. In addition, practitioners often do not receive training and contents of MSA training remain largely undefined, even though there are challenges in finding or developing relevant technical expertise. In this research, we determine the different technical roles that are required in MSAs, along with their detailed competences. We use public online forums (e.g., StackOverflow), where developers share technical knowledge. We analyze 13,517 public profiles of software engineers, deriving their technical competences. Our taxonomy of technical competences in MSAs, contains 11 competences clusters, organized in 3 collections of competences — Web Technologies, DevOps, and Data Technologies. In addition, we derive the roles of microservice practitioners and the characteristics of their roles. Our findings organize the technical competences of MSAs practitioners and determine the training topics and combination of topics that can prepare engineers for MSAs.
@article{ayas:24:1, title = {An empirical investigation on the competences and roles of practitioners in Microservices-based Architectures}, journal = {Journal of Systems and Software}, volume = {213}, pages = {112055}, year = {2024}, issn = {0164-1212}, doi = {https://doi.org/10.1016/j.jss.2024.112055}, author = {{Michael Ayas}, Hamdy and Hebig, Regina and Leitner, Philipp}, keywords = {Microservices, Technical competences, Practitioners profiles, Developer roles, Stackoverflow}, }

CHASE @ ICSE 2024

The Roles, Responsibilities, and Skills of Engineers in the Era of Microservices-Based Architectures

Hamdy Michael Ayas, Regina Hebig, and Philipp Leitner

In Proceedings of the 16th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE 2023) , Lisbon, Portugal, 2024

Bib

@inproceedings{ayas:24,
  title = {{The Roles, Responsibilities, and Skills of Engineers in the Era of Microservices-Based Architectures}},
  booktitle = {{Proceedings of the 16th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE 2023)}},
  author = {Ayas, Hamdy Michael and Hebig, Regina and Leitner, Philipp},
  year = {2024},
  isbn = {9798400705335},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  doi = {10.1145/3641822.3641871},
  pages = {13–23},
  numpages = {11},
  keywords = {microservices, job market, responsibilities, skills, human-aspects},
  location = {Lisbon, Portugal},
}

ICPC @ ICSE 2024
The Impact of Compiler Warnings on Code Quality in C++ Projects

Albin Johansson, Carl Holmberg, Francisco Gomes De Oliveira Neto, and Philipp Leitner

In Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension , Lisbon, Portugal, 2024

Abs Bib Thesis

Modern compilers often offer a variety of warning flags, which developers can enable to get feedback on code that, while syntactically correct, may be problematic. In the case of C++, one example of such "correct but problematic" code is code that leads to undefined behavior (UB). The usage of compiler warnings has long been suspected as a way to decrease bugs and increase code quality. However, empirical evidence that supports this hypothesis is rare. In this study, we present evidence from a study of 127 open source C++ projects. We categorize their usage of compiler warnings into five groups based on which warning flags are being used, and analyse the relationship between compiler warnings and five quality metrics (bugs, critical issues, vulnerabilities, code smells, and technical debt) using Bayesian analysis. We conclude that, in general, compiler warnings indeed correlate with, and potentially cause, higher code quality, with the clearest impact being on the number of critical issues in a project. Using stricter warning flags expectantly correlates with higher code quality in our study objects. However, there are substantial differences between projects, which we attribute to the project’s individual development culture. That is, while warnings matter, other factors such as quality culture, are likely to be even more important to source code quality.
@inproceedings{johansson:24, author = {Johansson, Albin and Holmberg, Carl and Gomes De Oliveira Neto, Francisco and Leitner, Philipp}, title = {The Impact of Compiler Warnings on Code Quality in C++ Projects}, year = {2024}, isbn = {9798400705861}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, doi = {10.1145/3643916.3644410}, booktitle = {Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension}, pages = {270–279}, numpages = {10}, keywords = {compiler warnings, code quality, bayesian data analysis}, location = {Lisbon, Portugal}, series = {ICPC '24}, }
SIGMETRICS 2024
Machine Learning Systems are Bloated and Vulnerable

Huaifeng Zhang, Mohannad Alhanahnah, Fahmi Abdulqadir Ahmed, Dyako Fatih, Philipp Leitner, and Ahmed Ali-Eldin

Proceedings of SIGMETRICS, Feb 2024

Abs Bib PDF

Today’s software is bloated with both code and features that are not used by most users. This bloat is prevalent across the entire software stack, from operating systems and applications to containers. Containers are lightweight virtualization technologies used to package code and dependencies, providing portable, reproducible and isolated environments. For their ease of use, data scientists often utilize machine learning containers to simplify their workflow. However, this convenience comes at a cost: containers are often bloated with unnecessary code and dependencies, resulting in very large sizes. In this paper, we analyze and quantify bloat in machine learning containers. We develop MMLB, a framework for analyzing bloat in software systems, focusing on machine learning containers. MMLB measures the amount of bloat at both the container and package levels, quantifying the sources of bloat. In addition, MMLB integrates with vulnerability analysis tools and performs package dependency analysis to evaluate the impact of bloat on container vulnerabilities. Through experimentation with 15 machine learning containers from TensorFlow, PyTorch, and Nvidia, we show that bloat accounts for up to 80% of machine learning container sizes, increasing container provisioning times by up to 370% and exacerbating vulnerabilities by up to 99%.
@article{zhang:24, author = {Zhang, Huaifeng and Alhanahnah, Mohannad and Ahmed, Fahmi Abdulqadir and Fatih, Dyako and Leitner, Philipp and Ali-Eldin, Ahmed}, title = {Machine Learning Systems are Bloated and Vulnerable}, year = {2024}, issue_date = {March 2024}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {8}, number = {1}, doi = {10.1145/3639032}, journal = {Proceedings of SIGMETRICS}, month = feb, articleno = {6}, numpages = {30}, keywords = {machine learning systems, software debloating}, }

2023

Big Data 2023
Batch Mode Deep Active Learning for Regression on Graph Data

P. Samoaa, L. Aronsson, P. Leitner, and M. Chehreghani

In 2023 IEEE International Conference on Big Data (BigData) , Dec 2023

Abs Bib PDF

Acquiring labelled data for machine learning tasks, for example, for software performance prediction, remains a resource-intensive task. This study extends our previous work by introducing a batch-mode deep active learning approach tailored for regression in graph-structured data. Our framework leverages the source code conversion into Flow Augmented-AST graphs (FA-AST), subsequently utilizing both supervised and unsupervised graph embeddings. In contrast to single-instance querying, the batch-mode paradigm adaptively selects clusters of unlabeled data for labelling. We deploy an array of base kernels, kernel transformations, and selection methods, informed by both Bayesian and non-Bayesian strategies, to enhance the sample efficiency of neural network regression. Our experimental evaluation, conducted on multiple real-world software performance datasets, demonstrates the efficacy of the batch mode deep active learning approach in achieving robust performance with a reduced labelling budget. The methodology scales effectively to larger datasets and requires minimal alterations to existing neural network architectures.
@inproceedings{samoaa:23, author = {Samoaa, P. and Aronsson, L. and Leitner, P. and Chehreghani, M.}, booktitle = {2023 IEEE International Conference on Big Data (BigData)}, title = {Batch Mode Deep Active Learning for Regression on Graph Data}, year = {2023}, volume = {}, issn = {}, pages = {5904-5913}, keywords = {source coding;neural networks;software performance;machine learning;gaussian processes;computer architecture;big data}, doi = {10.1109/BigData59044.2023.10386685}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, month = dec, }
SOSE 2023
The Perceived Impact and Sequence of Activities When Transitioning to Microservices

H. Ayas, P. Leitner, and R. Hebig

In 2023 IEEE International Conference on Service-Oriented System Engineering (SOSE) , Jul 2023

(short paper)

Abs Bib

Microservices migrations are often present in organizations that want to modernize and integrate their software systems. There is little empirical evidence showing how migration projects take place. Investigating migration activities and the perceived impact of migrations from practitioners is important to understand how migration projects materialize. In this study, we ask 54 practitioners about their views on specific aspects of microservices migrations. Specifically, we derive the sequence of migration activities as well as the perceived impact of microservices on the development process. Database refactoring, back-end refactoring, and setting up DevOps tend to take place before front-end refactoring, setting up communication patterns, and splitting teams. In addition, aligning teams with profitable value propositions, improving the testing process, and having fewer dependencies between teams are among the prominent impact areas of microservices. Our results call for further empirical research in understanding transitions toward MSAs.
@inproceedings{ayas:23:2, author = {Ayas, H. and Leitner, P. and Hebig, R.}, booktitle = {2023 IEEE International Conference on Service-Oriented System Engineering (SOSE)}, title = {The Perceived Impact and Sequence of Activities When Transitioning to Microservices}, year = {2023}, volume = {}, issn = {}, pages = {156-160}, keywords = {surveys;service-oriented systems engineering;databases;microservice architectures;organizations;software systems;testing}, doi = {10.1109/SOSE58276.2023.00026}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, month = jul, note = {(short paper)}, }

BotSE @ ICSE 2023

Towards Continuous Performance Assessment of Java Applications With PerfBot

Florian Markusse, Alexander Serebrenik, and Philipp Leitner

In 2023 IEEE/ACM 5th International Workshop on Bots in Software Engineering (BotSE) , Jul 2023

Bib

@inproceedings{markusse:23,
  author = {Markusse, Florian and Serebrenik, Alexander and Leitner, Philipp},
  booktitle = {2023 IEEE/ACM 5th International Workshop on Bots in Software Engineering (BotSE)},
  title = {Towards Continuous Performance Assessment of Java Applications With PerfBot},
  year = {2023},
  volume = {},
  number = {},
  pages = {6-8},
  keywords = {Java;Conferences;Software performance;Benchmark testing;Chatbots;Task analysis;Software engineering},
  doi = {10.1109/BotSE59190.2023.00009},
}

EMSE

An empirical study of the systemic and technical migration towards microservices

Hamdy Michael Ayas, Philipp Leitner, and Regina Hebig

Empirical Software Engineering, May 2023

Bib

@article{ayas:23,
  author = {Michael Ayas, Hamdy and Leitner, Philipp and Hebig, Regina},
  title = {An empirical study of the systemic and technical migration towards microservices},
  year = {2023},
  issue_date = {Jul 2023},
  publisher = {Kluwer Academic Publishers},
  address = {USA},
  volume = {28},
  number = {4},
  issn = {1382-3256},
  doi = {10.1007/s10664-023-10308-9},
  journal = {Empirical Software Engineering},
  month = may,
  numpages = {50},
  keywords = {Microservices, Migrations, Grounded theory, Empirical research},
}

TCC
Using Microbenchmark Suites to Detect Application Performance Changes

M. Grambow, D. Kovalev, C. Laaber, P. Leitner, and D. Bermbach

IEEE Transactions on Cloud Computing, Jul 2023

Abs Bib PDF

Software performance changes are costly and often hard to detect pre-release. Similar to software testing frameworks, either application benchmarks or microbenchmarks can be integrated into quality assurance pipelines to detect performance changes before releasing a new application version. Unfortunately, extensive benchmarking studies usually take several hours which is problematic when examining dozens of daily code changes in detail; hence, trade-offs have to be made. Optimized microbenchmark suites, which only include a small subset of the full suite, are a potential solution for this problem, given that they still reliably detect the majority of the application performance changes such as an increased request latency. It is, however, unclear whether microbenchmarks and application benchmarks detect the same performance problems and one can be a proxy for the other. In this paper, we explore whether microbenchmark suites can detect the same application performance changes as an application benchmark. For this, we run extensive benchmark experiments with both the complete and the optimized microbenchmark suites of the two time-series database systems InfluxDB and VictoriaMetrics and compare their results to the results of corresponding application benchmarks. We do this for 70 and 110 commits, respectively. Our results show that it is possible to detect application performance changes using an optimized microbenchmark suite if frequent false-positive alarms can be tolerated.
@article{grambow:22, author = {Grambow, M. and Kovalev, D. and Laaber, C. and Leitner, P. and Bermbach, D.}, journal = {IEEE Transactions on Cloud Computing}, title = {Using Microbenchmark Suites to Detect Application Performance Changes}, year = {2023}, volume = {11}, number = {03}, issn = {2168-7161}, pages = {2575-2590}, keywords = {benchmark testing;codes;cloud computing;reliability;software;pipelines;mimics}, doi = {10.1109/TCC.2022.3217947}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, month = jul, }
TSE
Automated Generation and Evaluation of JMH Microbenchmark Suites From Unit Tests

Mostafa Jangali, Yiming Tang, Niclas Alexandersson, Philipp Leitner, Jinqiu Yang, and Weiyi Shang

IEEE Transactions on Software Engineering, Apr 2023

Abs Bib PDF

Performance is a crucial non-functional requirement of many software systems. Despite the widespread use of performance testing, developers still struggle to construct and evaluate the quality of performance tests. To address these two major challenges, we implement a framework, dubbed ju2jmh, to automatically generate performance microbenchmarks from JUnit tests and use mutation testing to study the quality of generated microbenchmarks. Specifically, we compare our ju2jmh generated benchmarks to manually written JMH benchmarks and to automatically generated JMH benchmarks using the AutoJMH framework, as well as directly measuring system performance with JUnit tests. For this purpose, we have conducted a study on three subjects (Rxjava, Eclipse-collections, and Zipkin) with sim∼454K source lines of code (SLOC), 2,417 JMH benchmarks (including manually written and generated AutoJMH benchmarks) and 35,084 JUnit tests. Our results show that the ju2jmh generated JMH benchmarks consistently outperform using the execution time and throughput of JUnit tests as a proxy of performance and JMH benchmarks automatically generated using the AutoJMH framework while being comparable to JMH benchmarks manually written by developers in terms of tests’ stability and ability to detect performance bugs. Nevertheless, ju2jmh benchmarks are able to cover more of the software applications than manually written JMH benchmarks during the microbenchmark execution. Furthermore, ju2jmh benchmarks are generated automatically, while manually written JMH benchmarks require many hours of hard work and attention; therefore our study can reduce developers’ effort to construct microbenchmarks. In addition, we identify three factors (too low test workload, unstable tests and limited mutant coverage) that affect a benchmark’s ability to detect performance bugs. To the best of our knowledge, this is the first study aimed at assisting developers in fully automated microbenchmark creation and assessing microbenchmark quality for performance testing.
@article{jangali:22, author = {Jangali, Mostafa and Tang, Yiming and Alexandersson, Niclas and Leitner, Philipp and Yang, Jinqiu and Shang, Weiyi}, title = {Automated Generation and Evaluation of JMH Microbenchmark Suites From Unit Tests}, year = {2023}, issue_date = {April 2023}, publisher = {IEEE Press}, volume = {49}, number = {4}, issn = {0098-5589}, doi = {10.1109/TSE.2022.3188005}, journal = {IEEE Transactions on Software Engineering}, month = apr, pages = {1704–1725}, numpages = {22}, }

2022

UCC 2022

CrossFit: Fine-grained Benchmarking of Serverless Application Performance across Cloud Providers

Joel Scheuner, Rui Deng, Jan-Philipp Steghöfer, and Philipp Leitner

In 2022 IEEE/ACM 15th International Conference on Utility and Cloud Computing (UCC) , Apr 2022

Bib PDF

@inproceedings{scheuner:22:2,
  author = {Scheuner, Joel and Deng, Rui and Steghöfer, Jan-Philipp and Leitner, Philipp},
  booktitle = {2022 IEEE/ACM 15th International Conference on Utility and Cloud Computing (UCC)},
  title = {CrossFit: Fine-grained Benchmarking of Serverless Application Performance across Cloud Providers},
  year = {2022},
  volume = {},
  number = {},
  pages = {51-60},
  keywords = {Performance evaluation;Cloud computing;Runtime;Instruments;Serverless computing;Tail;Standardization;serverless;FaaS;distributed tracing;observability;performance;benchmarking},
  doi = {10.1109/UCC56403.2022.00016},
}

PROFES 2022
TEP-GNN: Accurate Execution Time Prediction of Functional Tests Using Graph Neural Networks

Hazem Peter Samoaa, Antonio Longa, Mazen Mohamad, Morteza Haghir Chehreghani, and Philipp Leitner

In Product-Focused Software Process Improvement , Apr 2022

Abs Bib PDF

Predicting the performance of production code prior to actual execution is known to be highly challenging. In this paper, we propose a predictive model, dubbed TEP-GNN, which demonstrates that high-accuracy performance prediction is possible for the special case of predicting unit test execution times. TEP-GNN uses FA-ASTs, or flow-augmented ASTs, as a graph-based code representation approach, and predicts test execution times using a powerful graph neural network (GNN) deep learning model. We evaluate TEP-GNN using four real-life Java open source programs, based on 922 test files mined from the projects’ public repositories. We find that our approach achieves a high Pearson correlation of 0.789, considerable outperforming a baseline deep learning model. Our work demonstrates that FA-ASTs and GNNs are a feasible approach for predicting absolute performance values, and serves as an important intermediary step towards being able to predict the performance of arbitrary code prior to execution.
@inproceedings{samoaa:22:1, author = {Samoaa, Hazem Peter and Longa, Antonio and Mohamad, Mazen and Chehreghani, Morteza Haghir and Leitner, Philipp}, editor = {Taibi, Davide and Kuhrmann, Marco and Mikkonen, Tommi and Klünder, Jil and Abrahamsson, Pekka}, title = {TEP-GNN: Accurate Execution Time Prediction of Functional Tests Using Graph Neural Networks}, booktitle = {Product-Focused Software Process Improvement}, year = {2022}, publisher = {Springer International Publishing}, address = {Cham}, pages = {464--479}, isbn = {978-3-031-21388-5}, }

IC2E 2022

TriggerBench: A Performance Benchmark for Serverless Function Triggers

Joel Scheuner, Marcus Bertilsson, Oskar Grönqvist, Henrik Tao, Henrik Lagergren, Jan-Philipp Steghöfer, and Philipp Leitner

In 2022 IEEE International Conference on Cloud Engineering (IC2E) , Apr 2022

Bib PDF

@inproceedings{scheuner:22,
  author = {Scheuner, Joel and Bertilsson, Marcus and Grönqvist, Oskar and Tao, Henrik and Lagergren, Henrik and Steghöfer, Jan-Philipp and Leitner, Philipp},
  booktitle = {2022 IEEE International Conference on Cloud Engineering (IC2E)},
  title = {TriggerBench: A Performance Benchmark for Serverless Function Triggers},
  year = {2022},
  volume = {},
  number = {},
  pages = {96-103},
  keywords = {Runtime;Scalability;Buildings;Serverless computing;Tail;Benchmark testing;Delays;serverless;FaaS;triggers;distributed tracing;observability;performance;benchmarking},
  doi = {10.1109/IC2E55432.2022.00018},
}

SEAA 2022

An Empirical Analysis of Microservices Systems Using Consumer-Driven Contract Testing

Hamdy Michael Ayas, Hartmut Fischer, Philipp Leitner, and Francisco Gomes De Oliveira Neto

In 2022 48th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) , Apr 2022

Bib

@inproceedings{ayas:22,
  author = {Ayas, Hamdy Michael and Fischer, Hartmut and Leitner, Philipp and De Oliveira Neto, Francisco Gomes},
  booktitle = {2022 48th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)},
  title = {An Empirical Analysis of Microservices Systems Using Consumer-Driven Contract Testing},
  year = {2022},
  volume = {},
  number = {},
  pages = {92-99},
  keywords = {Shape;Microservice architectures;Computer architecture;Software;Best practices;Contracts;Testing;Microservice architecture;consumer-driven contract test;repository mining},
  doi = {10.1109/SEAA56994.2022.00022},
}

IEEE Software

Using Benchmarking Bots for Continuous Performance Assessment

Florian Markusse, Philipp Leitner, and Alexander Serebrenik

IEEE Software, Apr 2022

Bib PDF

@article{markusse:22,
  author = {Markusse, Florian and Leitner, Philipp and Serebrenik, Alexander},
  journal = {IEEE Software},
  title = {Using Benchmarking Bots for Continuous Performance Assessment},
  year = {2022},
  volume = {39},
  number = {5},
  pages = {50-55},
  keywords = {Benchmark testing;Bot (Internet);Software;Codes;Software development management;Manuals;Open source software},
  doi = {10.1109/MS.2022.3184430},
}

IET Software
A systematic mapping study of source code representation for deep learning in software engineering

Hazem Peter Samoaa, Firas Bayram, Pasquale Salza, and Philipp Leitner

IET Software, Apr 2022

Abs Bib

The usage of deep learning (DL) approaches for software engineering has attracted much attention, particularly in source code modelling and analysis. However, in order to use DL, source code needs to be formatted to fit the expected input form of DL models. This problem is known as source code representation. Source code can be represented via different approaches, most importantly, the tree-based, token-based, and graph-based approaches. We use a systematic mapping study to investigate i detail the representation approaches adopted in 103 studies that use DL in the context of software engineering. Thus, studies are collected from 2014 to 2021 from 14 different journals and 27 conferences. We show that each way of representing source code can provide a different, yet orthogonal view of the same source code. Thus, different software engineering tasks might require different (combinations of) code representation approaches, depending on the nature and complexity of the task. Particularly, we show that it is crucial to define whether the DL approach requires lexical, syntactical, or semantic code information. Our analysis shows that a wide range of different representations and combinations of representations (hybrid representations) are used to solve a wide range of common software engineering problems. However, we also observe that current research does not generally attempt to transfer existing representations or models to other studies even though there are other contexts in which these representations and models may also be useful. We believe that there is potential for more reuse and the application of transfer learning when applying DL to software engineering tasks.
@article{samoaa:22, author = {Samoaa, Hazem Peter and Bayram, Firas and Salza, Pasquale and Leitner, Philipp}, title = {A systematic mapping study of source code representation for deep learning in software engineering}, journal = {IET Software}, volume = {16}, number = {4}, pages = {351-385}, doi = {https://doi.org/10.1049/sfw2.12064}, eprint = {https://ietresearch.onlinelibrary.wiley.com/doi/pdf/10.1049/sfw2.12064}, year = {2022}, }
TSE
The State of Serverless Applications: Collection, Characterization, and Community Consensus

Simon Eismann, Joel Scheuner, Erwin van Eyk, Maximilian Schwinger, Johannes Grohmann, Nikolas Herbst, Cristina L. Abad, and Alexandru Iosup

IEEE Transactions on Software Engineering, Oct 2022

Abs Bib Supp

Over the last five years, all major cloud platform providers have increased their serverless offerings. Many early adopters report significant benefits for serverless-based over traditional applications, and many companies are considering moving to serverless themselves. However, currently there exist only few, scattered, and sometimes even conflicting reports on when serverless applications are well suited and what the best practices for their implementation are. We address this problem in the present study about the state of serverless applications. We collect descriptions of 89 serverless applications from open-source projects, academic literature, industrial literature, and domain-specific feedback. We analyze 16 characteristics that describe why and when successful adopters are using serverless applications, and how they are building them. We further compare the results of our characterization study to 10 existing, mostly industrial, studies and datasets; this allows us to identify points of consensus across multiple studies, investigate points of disagreement, and overall confirm the validity of our results. The results of this study can help managers to decide if they should adopt serverless technology, engineers to learn about current practices of building serverless applications, and researchers and platform providers to better understand the current landscape of serverless applications.
@article{10.1109/TSE.2021.3113940, author = {Eismann, Simon and Scheuner, Joel and Eyk, Erwin van and Schwinger, Maximilian and Grohmann, Johannes and Herbst, Nikolas and Abad, Cristina L. and Iosup, Alexandru}, title = {The State of Serverless Applications: Collection, Characterization, and Community Consensus}, year = {2022}, issue_date = {Oct. 2022}, publisher = {IEEE Press}, volume = {48}, number = {10}, issn = {0098-5589}, doi = {10.1109/TSE.2021.3113940}, journal = {IEEE Transactions on Software Engineering}, month = oct, pages = {4152–4166}, numpages = {15}, }

2021

PeerJ CS

Dependency Management Bots in Open-Source Systems - Prevalence and Adoption

Linda Erlenhov, Francisco Gomes Oliveira Neto, and Philipp Leitner

PeerJ Computer Science, Oct 2021

Bib Supp

@article{erlenhov:22,
  title = {{Dependency Management Bots in Open-Source Systems - Prevalence and Adoption}},
  journal = {PeerJ Computer Science},
  year = {2021},
  author = {Erlenhov, Linda and de Oliveira Neto, Francisco Gomes and Leitner, Philipp},
  volume = {8},
  pages = {e849},
  doi = {https://doi.org/10.7717/peerj-cs.849},
}

ESEM 2021
Facing the Giant: a Grounded Theory Study of Decision-Making in Microservices Migrations

Hamdy Michael Ayas, Philipp Leitner, and Regina Hebig

In Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) , Bari, Italy, Oct 2021

Abs Bib

Background: Microservices migrations are challenging and expensive projects with many decisions that need to be made in a multitude of dimensions. Existing research tends to focus on technical issues and decisions (e.g., how to split services). Equally important organizational or business issues and their relations with technical aspects often remain out of scope or on a high level of abstraction.Aims: In this study, we aim to holistically chart the decision-making that happens on all dimensions of a migration project towards microservices (including, but not limited to, the technical dimension).Method: We investigate 16 different migration cases in a grounded theory interview study, with 19 participants that recently migrated towards microservices. This study strongly focuses on the human aspects of a migration, through stakeholders and their decisions.Results: We identify 3 decision-making processes consisting of 22 decision-points and their alternative options. The decision-points are related to creating stakeholder engagement and assessing feasibility, technical implementation, and organizational restructuring.Conclusions: Our study provides an initial theory of decisionmaking in migrations to microservices. It also outfits practitioners with a roadmap of which decisions they should be prepared to make and at which point in the migration.
@inproceedings{ayas:21:2, author = {Ayas, Hamdy Michael and Leitner, Philipp and Hebig, Regina}, title = {Facing the Giant: a Grounded Theory Study of Decision-Making in Microservices Migrations}, year = {2021}, isbn = {9781450386654}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, doi = {10.1145/3475716.3475792}, booktitle = {Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)}, articleno = {16}, numpages = {11}, keywords = {migration, microservices, interview study, grounded theory, decision-making}, location = {Bari, Italy}, series = {ESEM '21}, }

EMSE

Applying test case prioritization to software microbenchmarks

Christoph Laaber, Harald C. Gall, and Philipp Leitner

Empirical Software Engineering, Oct 2021

Bib Supp

@article{laaber:21,
  author = {Laaber, Christoph and Gall, Harald C. and Leitner, Philipp},
  title = {Applying test case prioritization to software microbenchmarks},
  journal = {Empirical Software Engineering},
  volume = {26},
  number = {6},
  pages = {133},
  year = {2021},
  doi = {10.1007/s10664-021-10037-x},
}

PeerJ CS

Using application benchmark call graphs to quantify and improve the practical relevance of microbenchmark suites

Martin Grambow, Christoph Laaber, Philipp Leitner, and David Bermbach

PeerJ Computer Science, Oct 2021

Bib

@article{grambow:21,
  author = {Grambow, Martin and Laaber, Christoph and Leitner, Philipp and Bermbach, David},
  title = {Using application benchmark call graphs to quantify and improve the
                 practical relevance of microbenchmark suites},
  journal = {PeerJ Computer Science},
  volume = {7},
  pages = {e548},
  year = {2021},
  doi = {10.7717/peerj-cs.548},
}

SOFSEM 2021
A Pipeline for Measuring Brand Loyalty Through Social Media Mining

Hazem Samoaa, and Barbara Catania

In SOFSEM 2021: Theory and Practice of Computer Science , Oct 2021

Abs Bib

Enhancing customer relationships through social media is an area of high relevance for companies. To this aim, Social Business Intelligence (SBI) plays a crucial role by supporting companies in combining corporate data with user-generated content, usually available as textual clips on social media. Unfortunately, SBI research is often constrained by the lack of publicly-available, real-world data for experimental activities. In this paper, we describe our experience in extracting social data and processing them through an enrichment pipeline for brand analysis. As a first step, we collect texts from social media and we annotate them based on predefined metrics for brand analysis, using features such as sentiment and geolocation. Annotations rely on various learning and natural language processing approaches, including deep learning and geographical ontologies. Structured data obtained from the annotation process are then stored in a distributed data warehouse for further analysis. Preliminary results, obtained from the analysis of three well known ICT brands, using data gathered from Twitter, news portals, and Amazon product reviews, show that different evaluation metrics can lead to different outcomes, indicating that no single metric is dominant for all brand analysis use cases.
@inproceedings{samoaa:21, author = {Samoaa, Hazem and Catania, Barbara}, title = {A Pipeline for Measuring Brand Loyalty Through Social Media Mining}, booktitle = {SOFSEM 2021: Theory and Practice of Computer Science}, year = {2021}, publisher = {Springer International Publishing}, address = {Cham}, pages = {489--504}, isbn = {978-3-030-67731-2}, }

ICPE 2021

An Exploratory Study of the Impact of Parameterization on JMH Measurement Results in Open-Source Projects

Hazem Samoaa, and Philipp Leitner

In ICPE ’21: ACM/SPEC International Conference on Performance Engineering, Virtual Event, France, April 19-21, 2021 , Oct 2021

Bib

@inproceedings{samoaa:21:2,
  author = {Samoaa, Hazem and Leitner, Philipp},
  editor = {Bourcier, Johann and Jiang, Zhen Ming (Jack) and Bezemer, Cor{-}Paul and Cortellessa, Vittorio and Pompeo, Daniele Di and Varbanescu, Ana Lucia},
  title = {An Exploratory Study of the Impact of Parameterization on {JMH} Measurement
                 Results in Open-Source Projects},
  booktitle = {{ICPE} '21: {ACM/SPEC} International Conference on Performance Engineering,
                 Virtual Event, France, April 19-21, 2021},
  pages = {213--224},
  publisher = {{ACM}},
  year = {2021},
  doi = {10.1145/3427921.3450243},
}

IEEE Software

Serverless Applications: Why, When, and How?

Simon Eismann, Joel Scheuner, Erwin Eyk, Maximilian Schwinger, Johannes Grohmann, Nikolas Herbst, Cristina L. Abad, and Alexandru Iosup

IEEE Software, Oct 2021

Bib

@article{eismann:21,
  author = {Eismann, Simon and Scheuner, Joel and van Eyk, Erwin and Schwinger, Maximilian and Grohmann, Johannes and Herbst, Nikolas and Abad, Cristina L. and Iosup, Alexandru},
  journal = {IEEE Software},
  title = {Serverless Applications: Why, When, and How?},
  year = {2021},
  volume = {38},
  number = {1},
  pages = {32-39},
  doi = {10.1109/MS.2020.3023302},
}

TSE

What’s Wrong with My Benchmark Results? Studying Bad Practices in JMH Benchmarks

Diego Costa, Cor-Paul Bezemer, Philipp Leitner, and Artur Andrzejak

IEEE Transactions on Software Engineering, Oct 2021

Bib

@article{costa:19,
  author = {Costa, Diego and Bezemer, Cor-Paul and Leitner, Philipp and Andrzejak, Artur},
  journal = {IEEE Transactions on Software Engineering},
  title = {What's Wrong with My Benchmark Results? Studying Bad Practices in JMH Benchmarks},
  year = {2021},
  volume = {47},
  number = {7},
  pages = {1452-1467},
  doi = {10.1109/TSE.2019.2925345},
}

TSE

Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test Case Generation

Giovanni Grano, Christoph Laaber, Annibale Panichella, and Sebastiano Panichella

IEEE Transactions on Software Engineering, Oct 2021

Bib

@article{grano:19,
  author = {Grano, Giovanni and Laaber, Christoph and Panichella, Annibale and Panichella, Sebastiano},
  title = {Testing with Fewer Resources: An Adaptive Approach to Performance-Aware
                 Test Case Generation},
  journal = {{IEEE} Transactions on Software Engineering},
  volume = {47},
  number = {11},
  pages = {2332--2347},
  year = {2021},
  doi = {10.1109/TSE.2019.2946773},
}

2020

Topology-Aware Continuous Experimentation in Microservice-Based Applications

Gerald Schermann, Fábio Oliveira, Erik Wittern, and Philipp Leitner

In Service-Oriented Computing - 18th International Conference, ICSOC 2020, Dubai, United Arab Emirates, December 14-17, 2020, Proceedings , Oct 2020

Bib

@inproceedings{schermann:20,
  author = {Schermann, Gerald and Oliveira, F{\'{a}}bio and Wittern, Erik and Leitner, Philipp},
  editor = {Kafeza, Eleanna and Benatallah, Boualem and Martinelli, Fabio and Hacid, Hakim and Bouguettaya, Athman and Motahari, Hamid},
  title = {Topology-Aware Continuous Experimentation in Microservice-Based Applications},
  booktitle = {Service-Oriented Computing - 18th International Conference, {ICSOC}
                 2020, Dubai, United Arab Emirates, December 14-17, 2020, Proceedings},
  series = {Lecture Notes in Computer Science},
  volume = {12571},
  pages = {19--35},
  publisher = {Springer},
  year = {2020},
  doi = {10.1007/978-3-030-65310-1\_2}
}

Function-as-a-Service performance evaluation: A multivocal literature review

Joel Scheuner, and Philipp Leitner

Journal of Systems and Software, Oct 2020

Bib

@article{scheuner:20,
  author = {Scheuner, Joel and Leitner, Philipp},
  title = {Function-as-a-Service performance evaluation: {A} multivocal literature
                 review},
  journal = {Journal of Systems and Software},
  volume = {170},
  pages = {110708},
  year = {2020},
  doi = {10.1016/j.jss.2020.110708}
}

FSE 2020

Dynamically Reconfiguring Software Microbenchmarks: Reducing Execution Time without Sacrificing Result Quality

Christoph Laaber, Stefan Würsten, Harald C. Gall, and Philipp Leitner

In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , Virtual Event, USA, Oct 2020

Bib

@inproceedings{laaber:20,
  author = {Laaber, Christoph and W\"{u}rsten, Stefan and Gall, Harald C. and Leitner, Philipp},
  title = {Dynamically Reconfiguring Software Microbenchmarks: Reducing Execution Time without Sacrificing Result Quality},
  year = {2020},
  isbn = {9781450370431},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  doi = {10.1145/3368089.3409683},
  booktitle = {Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
  pages = {989–1001},
  numpages = {13},
  keywords = {JMH, configuration, software benchmarking, performance testing},
  location = {Virtual Event, USA},
  series = {ESEC/FSE 2020},
}

FSE 2020

An Empirical Study of Bots in Software Development: Characteristics and Challenges from a Practitioner’s Perspective

Linda Erlenhov, Francisco Gomes de Oliveira Neto, and Philipp Leitner

In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , Virtual Event, USA, Oct 2020

Bib

@inproceedings{erlenhov:20,
  author = {Erlenhov, Linda and Neto, Francisco Gomes de Oliveira and Leitner, Philipp},
  title = {An Empirical Study of Bots in Software Development: Characteristics and Challenges from a Practitioner’s Perspective},
  year = {2020},
  isbn = {9781450370431},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  doi = {10.1145/3368089.3409680},
  booktitle = {Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
  pages = {445–455},
  numpages = {11},
  keywords = {Software engineering, Software bot, Empirical study},
  location = {Virtual Event, USA},
  series = {ESEC/FSE 2020},
}

Beyond Microbenchmarks: The SPEC-RG Vision for a Comprehensive Serverless Benchmark

Erwin Eyk, Joel Scheuner, Simon Eismann, Cristina L. Abad, and Alexandru Iosup

In Companion of the ACM/SPEC International Conference on Performance Engineering , Apr 2020

Bib

@inproceedings{eyk:20,
  author = {van Eyk, Erwin and Scheuner, Joel and Eismann, Simon and Abad, Cristina L. and Iosup, Alexandru},
  booktitle = {Companion of the ACM/SPEC International Conference on
  Performance Engineering},
  doi = {10.1145/3375555.3384381},
  month = apr,
  pages = {26--31},
  publisher = {ACM/SPEC},
  series = {ICPE '20},
  title = {Beyond Microbenchmarks: The SPEC-RG Vision for a
  Comprehensive Serverless Benchmark},
  year = {2020}
}

2019

Studying the impact of CI on pull request delivery time in open source projects — a conceptual replication

Yunfang Guo, and Philipp Leitner

PeerJ Computer Science, Apr 2019

Bib

@article{guo:19,
  title = {{Studying the impact of CI on pull request delivery time in open source projects — a conceptual replication}},
  journal = {PeerJ Computer Science},
  year = {2019},
  author = {Guo, Yunfang and Leitner, Philipp},
  doi = {10.7717/peerj-cs.245}
}

Transpiling Applications into Optimized Serverless Orchestrations

Joel Scheuner, and Philipp Leitner

In 2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W) , Jun 2019

2nd Workshop on Hot Topics in Cloud Computing Performance (HotCloudPerf)

Bib

@inproceedings{scheuner:19:3,
  author = {Scheuner, Joel and Leitner, Philipp},
  booktitle = {2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W)},
  doi = {10.1109/FAS-W.2019.00031},
  month = jun,
  pages = {72--73},
  publisher = {IEEE},
  title = {Transpiling Applications into Optimized Serverless Orchestrations},
  year = {2019},
  note = {2nd Workshop on Hot Topics in Cloud Computing Performance (HotCloudPerf)}
}

Tutorial – Performance Benchmarking of Infrastructure-as-a-Service (IaaS) Clouds with Cloud WorkBench

Joel Scheuner, and Philipp Leitner

In 2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W) , Jun 2019

Tutorial co-located with ICAC/SASO

Bib

@inproceedings{scheuner:19:2,
  author = {Scheuner, Joel and Leitner, Philipp},
  booktitle = {2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W)},
  doi = {10.1109/FAS-W.2019.00070},
  month = jun,
  pages = {257--258},
  publisher = {IEEE},
  title = {Tutorial -- Performance Benchmarking of Infrastructure-as-a-Service (IaaS) Clouds with Cloud WorkBench},
  year = {2019},
  note = {Tutorial co-located with ICAC/SASO}
}

Current and Future Bots in Software Development

Linda Erlenhov, Francisco Gomes, Riccardo Scandariato, and Philipp Leitner

In Proceedings of the 1st International Workshop on Bots in Software Engineering (BotSE) at ICSE 2019 , Jun 2019

Bib

@inproceedings{erlenhov:19,
  title = {{Current and Future Bots in Software Development}},
  booktitle = {{Proceedings of the 1st International Workshop on Bots in Software
                 Engineering (BotSE) at ICSE 2019}},
  author = {Erlenhov, Linda and Gomes, Francisco and Scandariato, Riccardo and Leitner, Philipp},
  pages = {7--11},
  publisher = {{IEEE} / {ACM}},
  year = {2019},
  doi = {10.1109/BotSE.2019.00009}
}

Cloud Futurology

Blesson Varghese, Philipp Leitner, Suprio Ray, Kyle Chard, Adam Barker, Yehia Elkhatib, Herry Herry, Cheol-Ho Hong, Jeremy Singer, Fung Po Tso, Eiko Yoneki, and Mohamed-Faten Zhani

IEEE Computer, Jun 2019

Bib

@article{varghese:19,
  title = {{Cloud Futurology}},
  journal = {{IEEE} Computer},
  author = {Varghese, Blesson and Leitner, Philipp and Ray, Suprio and Chard, Kyle and Barker, Adam and Elkhatib, Yehia and Herry, Herry and Hong, Cheol-Ho and Singer, Jeremy and Tso, Fung Po and Yoneki, Eiko and and Mohamed-Faten Zhani},
  volume = {52},
  number = {9},
  pages = {68--77},
  year = {2019},
  doi = {10.1109/MC.2019.2895307}
}

Tutorial: Performance Benchmarking of Infrastructure-as-a-Service (IaaS) Clouds with Cloud WorkBench

Joel Scheuner, and Philipp Leitner

In Companion Proceedings of the 10th ACM/SPEC International Conference on Performance Engineering (ICPE) , Mumbai, India, Jun 2019

Bib

@inproceedings{scheuner:19,
  title = {{Tutorial: Performance Benchmarking of Infrastructure-as-a-Service (IaaS) Clouds with Cloud WorkBench}},
  booktitle = {Companion Proceedings of the 10th ACM/SPEC International Conference on Performance Engineering (ICPE)},
  year = {2019},
  author = {Scheuner, Joel and Leitner, Philipp},
  doi = {10.1145/3302541.3310294},
  keywords = {benchmarking, cloud computing, performance},
  isbn = {978-1-4503-6286-3},
  location = {Mumbai, India},
  pages = {53--56}
}

Software microbenchmarking in the cloud. How bad is it really?

Christoph Laaber, Joel Scheuner, and Philipp Leitner

Empirical Software Engineering, Apr 2019

Abs Bib

Rigorous performance engineering traditionally assumes measuring on bare-metal environments to control for as many confounding factors as possible. Unfortunately, some researchers and practitioners might not have access, knowledge, or funds to operate dedicated performance-testing hardware, making public clouds an attractive alternative. However, shared public cloud environments are inherently unpredictable in terms of the system performance they provide. In this study, we explore the effects of cloud environments on the variability of performance test results and to what extent slowdowns can still be reliably detected even in a public cloud. We focus on software microbenchmarks as an example of performance tests and execute extensive experiments on three different well-known public cloud services (AWS, GCE, and Azure) using three different cloud instance types per service. We also compare the results to a hosted bare-metal offering from IBM Bluemix. In total, we gathered more than 4.5 million unique microbenchmarking data points from benchmarks written in Java and Go. We find that the variability of results differs substantially between benchmarks and instance types (by a coefficient of variation from 0.03% to >\thinspace100%). However, executing test and control experiments on the same instances (in randomized order) allows us to detect slowdowns of 10% or less with high confidence, using state-of-the-art statistical tests (i.e., Wilcoxon rank-sum and overlapping bootstrapped confidence intervals). Finally, our results indicate that Wilcoxon rank-sum manages to detect smaller slowdowns in cloud environments.
@article{laaber:19a, author = {Laaber, Christoph and Scheuner, Joel and Leitner, Philipp}, title = {Software microbenchmarking in the cloud. How bad is it really?}, journal = {Empirical Software Engineering}, year = {2019}, month = apr, day = {17}, issn = {1573-7616}, doi = {10.1007/s10664-019-09681-1}, publisher = {Springer}, keywords = {performance testing, microbenchmarking, cloud, performance-regression detection}, numpages = {40}, abbrev = {EMSE}, }

Continuous Software Performance Assessment: Detecting Performance Problems of Software Libraries on Every Build

Christoph Laaber

In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis , Beijing, China, Apr 2019

Bib

@inproceedings{laaber:19b,
  author = {Laaber, Christoph},
  title = {Continuous Software Performance Assessment: Detecting Performance Problems of Software Libraries on Every Build},
  booktitle = {Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis},
  series = {ISSTA 2019},
  year = {2019},
  isbn = {978-1-4503-6224-5},
  location = {Beijing, China},
  pages = {410--414},
  numpages = {5},
  doi = {10.1145/3293882.3338982},
  acmid = {3338982},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {cloud, continuous integration, performance testing}
}

A Mixed-Method Empirical Study of Function-as-a-Service Software Development in Industrial Practice

Philipp Leitner, Erik Wittern, Josef Spillner, and Waldemar Hummer

Journal of Systems and Software, Apr 2019

Bib

@article{leitner:18,
  title = {{A Mixed-Method Empirical Study of Function-as-a-Service Software Development in Industrial Practice}},
  journal = {Journal of Systems and Software},
  volume = {149},
  pages = {340 - 359},
  year = {2019},
  issn = {0164-1212},
  doi = {https://doi.org/10.1016/j.jss.2018.12.013},
  author = {Leitner, Philipp and Wittern, Erik and Spillner, Josef and Hummer, Waldemar},
  keywords = {Cloud computing, Serverless, Function-as-a-Service, Empirical research}
}

ICSE 2019

Interactive Production Performance Feedback in the IDE

Jürgen Cito, Philipp Leitner, Martin Rinard, and Harald Gall

In Proceedings of the 41st International Conference on Software Engineering (ICSE) , Apr 2019

Bib PDF

@inproceedings{cito:19,
  author = {Cito, Jürgen and Leitner, Philipp and Rinard, Martin and Gall, Harald},
  title = {Interactive Production Performance Feedback in the IDE},
  booktitle = {{Proceedings of the 41st International Conference on Software Engineering (ICSE)}},
  pages = {971--981},
  publisher = {{IEEE} / {ACM}},
  year = {2019},
  doi = {10.1109/ICSE.2019.00102},
}

Cachematic -– Automatic Invalidation in Application-Level Caching Systems

Viktor Holmqvist, Jonathan Nilsfors, and Philipp Leitner

In Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering (ICPE) , Apr 2019

Bib

@inproceedings{holmqvist:19,
  author = {Holmqvist, Viktor and Nilsfors, Jonathan and Leitner, Philipp},
  title = {Cachematic -– Automatic Invalidation in Application-Level Caching Systems},
  booktitle = {Proceedings of the 2019 {ACM/SPEC} International Conference on Performance
                  Engineering {(ICPE)}},
  pages = {167--178},
  publisher = {{ACM}},
  year = {2019},
}

2018

Visualizing Test Diversity to Support Test Optimisation

F. G. De Oliveira Neto, R. Feldt, L. Erlenhov, and J. B. D. S. Nunes

In 2018 25th Asia-Pacific Software Engineering Conference (APSEC) , Dec 2018

Bib

@inproceedings{neto:18,
  author = {{De Oliveira Neto}, F. G. and {Feldt}, R. and {Erlenhov}, L. and {Nunes}, J. B. D. S.},
  booktitle = {2018 25th Asia-Pacific Software Engineering Conference (APSEC)},
  title = {Visualizing Test Diversity to Support Test Optimisation},
  year = {2018},
  volume = {},
  number = {},
  pages = {149-158},
  keywords = {data visualisation;fault diagnosis;program testing;test optimisation;test suites;cost-effective testing;test cases;diversity information;diversity-based approaches;test requirements;test scripts;test diversity data;test similarity maps;pair-wise diversity calculations;test repositories;maintenance;optimisation activities;Optimization;Testing;Indexes;Data visualization;Fault detection;Maintenance engineering;Subspace constraints;Software Testing;Diversity;Search based Software Testing;Empirical Study},
  doi = {10.1109/APSEC.2018.00029},
  issn = {},
  month = dec,
}

Structured Information on State and Evolution of Dockerfiles on Github

Gerald Schermann, Sali Zumberi, and Jürgen Cito

In Proceedings of the 15th International Conference on Mining Software Repositories , Gothenburg, Sweden, Dec 2018

Bib

@inproceedings{schermann:18:4,
  author = {Schermann, Gerald and Zumberi, Sali and Cito, J\"{u}rgen},
  title = {Structured Information on State and Evolution of Dockerfiles on Github},
  booktitle = {Proceedings of the 15th International Conference on Mining Software Repositories},
  series = {MSR '18},
  year = {2018},
  isbn = {978-1-4503-5716-6},
  location = {Gothenburg, Sweden},
  pages = {26--29},
  numpages = {4},
  doi = {10.1145/3196398.3196456},
  acmid = {3196456},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {containers, docker, github, mining software repositories}
}

Search-Based Scheduling of Experiments in Continuous Deployment

G. Schermann, and P. Leitner

In 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME) , Sep 2018

Bib

@inproceedings{schermann:18:3,
  author = {Schermann, G. and Leitner, P.},
  booktitle = {2018 IEEE International Conference on Software Maintenance and Evolution (ICSME)},
  title = {Search-Based Scheduling of Experiments in Continuous Deployment},
  year = {2018},
  pages = {485-495},
  keywords = {genetic algorithms;parallel processing;scheduling;search problems;continuous deployment;production environments;user assignments;optimization problem;parallel execution;genetic algorithm;local search;genetic implementation;search-based scheduling;Schedules;Companies;Testing;Google;Genetic algorithms;Simulated annealing;continuous experimentation;search based software engineering;scheduling;release engineering},
  doi = {10.1109/ICSME.2018.00059},
  issn = {2576-3148},
  month = sep
}

Estimating Cloud Application Performance Based on Micro-Benchmark Profiling

Joel Scheuner, and Philipp Leitner

In 2018 IEEE 11th International Conference on Cloud Computing (CLOUD) , Jul 2018

Bib

@inproceedings{scheuner:18:2,
  author = {Scheuner, Joel and Leitner, Philipp},
  booktitle = {2018 IEEE 11th International Conference on Cloud Computing (CLOUD)},
  title = {Estimating Cloud Application Performance Based on Micro-Benchmark Profiling},
  year = {2018},
  pages = {90-97},
  keywords = {benchmark testing;cloud computing;Amazon EC2;Web serving application;real-world applications;synthetic microbenchmarks;service selection;cloud computing market;microbenchmark profiling;cloud application performance;scientific computing application;cloud service performance;cloud provider;cloud benchmarking methodology;Benchmark testing;Cloud computing;Measurement;Random access memory;Computational modeling;Time factors;Predictive models;cloud computing;benchmarking;performance;micro benchmark;application benchmark;Web application;performance prediction},
  doi = {10.1109/CLOUD.2018.00019},
  issn = {2159-6190},
  month = jul
}

An Evaluation of Open-source Software Microbenchmark Suites for Continuous Performance Assessment

Christoph Laaber, and Philipp Leitner

In Proceedings of the 15th International Conference on Mining Software Repositories , Gothenburg, Sweden, Jul 2018

Bib

@inproceedings{laaber:18,
  author = {Laaber, Christoph and Leitner, Philipp},
  title = {An Evaluation of Open-source Software Microbenchmark Suites for Continuous Performance Assessment},
  booktitle = {Proceedings of the 15th International Conference on Mining Software Repositories},
  series = {MSR '18},
  year = {2018},
  isbn = {978-1-4503-5716-6},
  location = {Gothenburg, Sweden},
  pages = {119--130},
  numpages = {12},
  doi = {10.1145/3196398.3196407},
  acmid = {3196407},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {Go, Java, continuous integration, empirical study, microbenchmarking, software performance testing}
}

We’re Doing It Live: A Multi-Method Empirical Study on Continuous Experimentation

Gerald Schermann, Jürgen Cito, Philipp Leitner, Uwe Zdun, and Harald C. Gall

Journal of Information and Software Technology, Jul 2018

Bib

@article{schermann:18:2,
  author = {Schermann, Gerald and Cito, Jürgen and Leitner, Philipp and Zdun, Uwe and Gall, Harald C.},
  title = {{We’re Doing It Live: A Multi-Method Empirical Study on Continuous Experimentation}},
  journal = {{Journal of Information and Software Technology}},
  year = {2018},
  volume = { 99 },
  pages = {41--57},
  doi = {https://doi.org/10.1016/j.infsof.2018.02.010},
}

A Cloud Benchmark Suite Combining Micro and Applications Benchmarks

Joel Scheuner, and Philipp Leitner

In Companion of the 2018 ACM/SPEC International Conference on Performance Engineering , Berlin, Germany, Jul 2018

Bib

@inproceedings{scheuner:18,
  author = {Scheuner, Joel and Leitner, Philipp},
  title = {A Cloud Benchmark Suite Combining Micro and Applications Benchmarks},
  booktitle = {Companion of the 2018 ACM/SPEC International Conference on Performance Engineering},
  series = {ICPE '18},
  year = {2018},
  isbn = {978-1-4503-5629-9},
  location = {Berlin, Germany},
  pages = {161--166},
  numpages = {6},
  doi = {10.1145/3185768.3186286},
  acmid = {3186286},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {application benchmark, benchmarking, cloud computing, micro benchmark, performance, web application}
}

PerformanceHat: Augmenting Source Code with Runtime Performance Traces in the IDE

Jürgen Cito, Philipp Leitner, Christian Bosshard, Markus Knecht, Genc Mazlami, and Harald C. Gall

In Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings , Gothenburg, Sweden, Jul 2018

Bib

@inproceedings{cito:18,
  author = {Cito, J\"{u}rgen and Leitner, Philipp and Bosshard, Christian and Knecht, Markus and Mazlami, Genc and Gall, Harald C.},
  title = {PerformanceHat: Augmenting Source Code with Runtime Performance Traces in the IDE},
  booktitle = {Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings},
  series = {ICSE '18},
  year = {2018},
  isbn = {978-1-4503-5663-3},
  location = {Gothenburg, Sweden},
  pages = {41--44},
  numpages = {4},
  doi = {10.1145/3183440.3183481},
  acmid = {3183481},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {development workflow, program analysis, software performance engineering}
}

Continuous Experimentation: Challenges, Implementation Techniques, and Current Research

G. Schermann, J. Cito, and P. Leitner

IEEE Software, Mar 2018

Bib

@article{schermann:18,
  author = {Schermann, G. and Cito, J. and Leitner, P.},
  journal = {IEEE Software},
  title = {Continuous Experimentation: Challenges, Implementation Techniques, and Current Research},
  year = {2018},
  volume = {35},
  number = {2},
  pages = {26-31},
  keywords = {formal specification;Internet;program testing;systems analysis;requirements engineering;Web-based systems;Software engineering;Routing;Software testing;Data science;Computer architecture;Software product lines;Product life cycle management;release engineering;continuous experimentation;feature toggles;traffic routing;regression-driven experiments;business-driven experiments;software development;software engineering},
  doi = {10.1109/MS.2018.111094748},
  issn = {0740-7459},
  month = mar
}

2017

Report from GI-Dagstuhl Seminar 16394: Software Performance Engineering in the DevOps World

Andre Hoorn, Pooyan Jamshidi, Philipp Leitner, and Ingo Weber

CoRR, Mar 2017

Bib

@article{vanhoorn:18,
  author = {van Hoorn, Andre and Jamshidi, Pooyan and Leitner, Philipp and Weber, Ingo},
  title = {{Report from GI-Dagstuhl Seminar 16394: Software Performance Engineering in the DevOps World}},
  journal = {CoRR},
  year = {2017},
  volume = { nn },
  pages = {nn},
}

Optimized IoT service placement in the fog

Olena Skarlat, Matteo Nardelli, Stefan Schulte, Michael Borkowski, and Philipp Leitner

Service Oriented Computing and Applications, Dec 2017

Bib

@article{skarlat:17,
  author = {Skarlat, Olena and Nardelli, Matteo and Schulte, Stefan and Borkowski, Michael and Leitner, Philipp},
  title = {Optimized IoT service placement in the fog},
  journal = {Service Oriented Computing and Applications},
  year = {2017},
  month = dec,
  day = {01},
  volume = {11},
  number = {4},
  pages = {427--443},
  issn = {1863-2394},
  doi = {10.1007/s11761-017-0219-8},
}

A Tale of CI Build Failures: an Open Source and a Financial Organization Perspective

Carmine Vassallo, Gerald Schermann, Fiorella Zampetti, Daniele Romano, Philipp Leitner, Andy Zaidman, Massimiliano Di Penta, and Sebastiano Panichella

In Proceedings of the 33rd IEEE International Conference on Software Maintenance and Evolution (ICSME) , Dec 2017

Bib

@inproceedings{vassallo:17,
  author = {Vassallo, Carmine and Schermann, Gerald and Zampetti, Fiorella and Romano, Daniele and Leitner, Philipp and Zaidman, Andy and Penta, Massimiliano Di and Panichella, Sebastiano},
  title = {A Tale of CI Build Failures: an Open Source and a Financial Organization Perspective},
  booktitle = {Proceedings of the 33rd IEEE International Conference on Software Maintenance and Evolution (ICSME)},
  year = {2017},
  pages = {183-193},
  doi = {10.1109/ICSME.2017.67},
}

Extraction of Microservices from Monolithic Software Architectures

Genc Mazlami, Jürgen Cito, and Philipp Leitner

In Proceedings of the 24th IEEE International Conference on Web Services (ICWS) - Applications Track , Dec 2017

Bib

@inproceedings{mazlami:17,
  author = {Mazlami, Genc and Cito, Jürgen and Leitner, Philipp},
  title = {Extraction of Microservices from Monolithic Software Architectures},
  booktitle = {Proceedings of the 24th IEEE International Conference on Web Services (ICWS) - Applications Track},
  year = {2017},
  pages = {524--531},
  doi = {10.1109/ICWS.2017.61}
}

An Empirical Analysis of Build Failures in the Continuous Integration Workflows of Java-Based Open-Source Software

Thomas Rausch, Waldemar Hummer, Philipp Leitner, and Stefan Schulte

In Proceedings of the 14th International Conference on Mining Software Repositories , Buenos Aires, Argentinia, Dec 2017

Bib

@inproceedings{rausch:17,
  author = {Rausch, Thomas and Hummer, Waldemar and Leitner, Philipp and Schulte, Stefan},
  title = {An Empirical Analysis of Build Failures in the Continuous Integration Workflows of Java-Based Open-Source Software},
  booktitle = {Proceedings of the 14th International Conference on Mining Software Repositories},
  series = {MSR'17},
  year = {2017},
  pages = {345--355},
  location = {Buenos Aires, Argentinia},
  publisher = {ACM},
  address = {New York, NY, USA},
}

An Empirical Analysis of the Docker Container Ecosystem on GitHub

Jürgen Cito, Gerald Schermann, Erik Wittern, Philipp Leitner, Sali Zumberi, and Harald Gall

In Proceedings of the 14th International Conference on Mining Software Repositories , Buenos Aires, Argentinia, Dec 2017

Bib

@inproceedings{cito:17,
  author = {Cito, Jürgen and Schermann, Gerald and Wittern, Erik and Leitner, Philipp and Zumberi, Sali and Gall, Harald},
  title = {An Empirical Analysis of the Docker Container Ecosystem on GitHub},
  booktitle = {Proceedings of the 14th International Conference on Mining Software Repositories},
  series = {MSR'17},
  year = {2017},
  pages = {323--333},
  location = {Buenos Aires, Argentinia},
  publisher = {ACM},
  address = {New York, NY, USA},
}

(h | g)opper: Performance History Mining and Analysis

Christoph Laaber, and Philipp Leitner

In Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE) - Poster and Demonstration Track , L’Aquila, Italy, Dec 2017

Bib

@inproceedings{laaber:17,
  author = {Laaber, Christoph and Leitner, Philipp},
  title = {{(h | g)opper: Performance History Mining and Analysis}},
  booktitle = {Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE) - Poster and Demonstration Track},
  year = {2017},
  doi = {10.1145/3030207.3053662},
  acmid = {3053662},
  publisher = {ACM},
  address = {New York, NY, USA},
  isbn = {978-1-4503-4404-3},
  location = {L'Aquila, Italy},
  pages = {167--168}
}

An Approach and Case Study of Cloud Instance Type Selection for Multi-Tier Web Applications

Christian Davatz, Christian Inzinger, Joel Scheuner, and Philipp Leitner

In Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing , Madrid, Spain, Dec 2017

Bib

@inproceedings{davatz:17,
  author = {Davatz, Christian and Inzinger, Christian and Scheuner, Joel and Leitner, Philipp},
  title = {An Approach and Case Study of Cloud Instance Type Selection for Multi-Tier Web Applications},
  booktitle = {Proceedings of the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing},
  series = {CCGrid '17},
  year = {2017},
  isbn = {978-1-5090-6610-0},
  location = {Madrid, Spain},
  pages = {534--543},
  numpages = {10},
  doi = {10.1109/CCGRID.2017.12},
  acmid = {3101184},
  publisher = {IEEE Press},
  address = {Piscataway, NJ, USA}
}

Context-Based Analytics – Establishing Explicit Links between Runtime Traces and Source Code

Jürgen Cito, Faìbio Oliveira, Philipp Leitner, Priya Nagpurkar, and Harald Gall

In Proceedings of the 38th International Conference on Software Engineering (ICSE’17) - Software Engineering in Practice (SEIP) Track , Dec 2017

Bib

@inproceedings{cito:20,
  author = {Cito, Jürgen and Oliveira, Faìbio and Leitner, Philipp and Nagpurkar, Priya and Gall, Harald},
  title = { Context-Based Analytics -- Establishing Explicit Links between Runtime Traces and Source Code},
  booktitle = {Proceedings of the 38th International Conference on Software Engineering (ICSE'17) - Software Engineering in Practice (SEIP) Track},
  year = {2017},
  pages = {193--202},
}

An Exploratory Study of the State of Practice of Performance Testing in Java-Based Open Source Projects

Philipp Leitner, and Cor-Paul Bezemer

In Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE) , Dec 2017

Bib

@inproceedings{leitner:17,
  author = {Leitner, Philipp and Bezemer, Cor-Paul},
  title = {An Exploratory Study of the State of Practice of Performance Testing in Java-Based Open Source Projects},
  booktitle = {Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE)},
  year = {2017},
  pages = {373--384},
  doi = {10.1145/2901739.2901774},
}

2016

Probr – A Generic and Passive WiFi Tracking System

Joel Scheuner, Genc Mazlami, Dominik Schöni, Sebastian Stephan, Alessandro De Carli, Thomas Bocek, and Burkhard Stiller

In Proceedings of the 41st IEEE Conference on Local Computer Networks (LCN) , Dec 2016

Bib

@inproceedings{scheuner:16probr,
  author = {Scheuner, Joel and Mazlami, Genc and Sch{\"o}ni, Dominik and Stephan, Sebastian and Carli, Alessandro De and Bocek, Thomas and Stiller, Burkhard},
  booktitle = {Proceedings of the 41st IEEE Conference on Local Computer Networks (LCN)},
  title = {{Probr -- A Generic and Passive WiFi Tracking System}},
  year = {2016},
  doi = {10.1109/LCN.2016.30},
}

Probr Demonstration – Visualizing Passive WiFi Data

Joel Scheuner, Genc Mazlami, Dominik Schöni, Sebastian Stephan, Alessandro De Carli, Thomas Bocek, and Burkhard Stiller

In Proceedings of the 41st IEEE Conference on Local Computer Networks (LCN) , Dec 2016

Bib

@inproceedings{scheuner:16probrdemo,
  author = {Scheuner, Joel and Mazlami, Genc and Sch{\"o}ni, Dominik and Stephan, Sebastian and Carli, Alessandro De and Bocek, Thomas and Stiller, Burkhard},
  booktitle = {Proceedings of the 41st IEEE Conference on Local Computer Networks (LCN)},
  title = {{Probr Demonstration -- Visualizing Passive WiFi Data}},
  year = {2016},
}

Resource Provisioning for IoT Services in the Fog

Olena Skarlat, Stefan Schulte, Michael Borkowski, and Philipp Leitner

In 9th IEEE International Conference on Service-Oriented Computing and Applications (SOCA) , Dec 2016

Bib

@inproceedings{skarlat:16,
  author = {Skarlat, Olena and Schulte, Stefan and Borkowski, Michael and Leitner, Philipp},
  title = {Resource Provisioning for IoT Services in the Fog},
  booktitle = {9th {IEEE} International Conference on Service-Oriented Computing
                and Applications ({SOCA})},
  pages = {32--39},
  year = {2016},
  doi = {10.1109/SOCA.2016.10}
}

Modelling and Managing Deployment Costs of Microservice-Based Cloud Applications

Philipp Leitner, Jürgen Cito, and Emanuel Stöckli

In Proceedings of the 9th IEEE/ACM International Conference on Utility and Cloud Computing (UCC) , Dec 2016

Bib

@inproceedings{leitner:16:2,
  author = {Leitner, Philipp and Cito, Jürgen and Stöckli, Emanuel},
  title = {Modelling and Managing Deployment Costs of Microservice-Based Cloud Applications},
  booktitle = {Proceedings of the 9th IEEE/ACM International Conference on Utility and Cloud Computing (UCC)},
  year = {2016},
  pages = {nn},
}

Bifrost - Supporting Continuous Deployment with Automated Enactment of Multi-Phase Live Testing Strategies

Gerald Schermann, Dominik Schöni, Philipp Leitner, and Harald C. Gall

In Proceedings of the 2016 ACM/IFIP/USENIX Middleware Conference, Best Student Paper Award , Trento, Italy, Dec 2016

Bib

@inproceedings{schermann:16:2,
  author = {Schermann, Gerald and Sch\"oni, Dominik and Leitner, Philipp and Gall, Harald C.},
  title = {Bifrost - Supporting Continuous Deployment with Automated Enactment of Multi-Phase Live Testing Strategies},
  booktitle = {Proceedings of the 2016 ACM/IFIP/USENIX Middleware Conference, <strong>Best Student Paper Award</strong>},
  year = {2016},
  location = {Trento, Italy},
  pages = {nn},
}

TemPerf: Temporal Correlation Between Performance Metrics and Source Code

Jürgen Cito, Genc Mazlami, and Philipp Leitner

In Proceedings of the 2nd International Workshop on Quality-Aware DevOps , Saarbrücken, Germany, Dec 2016

Bib

@inproceedings{cito:16,
  author = {Cito, Jürgen and Mazlami, Genc and Leitner, Philipp},
  title = {TemPerf: Temporal Correlation Between Performance Metrics and Source Code},
  booktitle = {Proceedings of the 2nd International Workshop on Quality-Aware DevOps},
  series = {QUDOS 2016},
  year = {2016},
  isbn = {978-1-4503-4411-1},
  location = {Saarbr\&\#252;cken, Germany},
  pages = {46--47},
  numpages = {2},
  doi = {10.1145/2945408.2945420},
  acmid = {2945420},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {DevOps, Developer Tools, Performance}
}

Towards Quality Gates in Continuous Delivery and Deployment

Gerald Schermann, Jürgen Cito, Philipp Leitner, and Harald C. Gall

In Proceedings of the 24TH IEEE International Conference on Program Comprehension (ICPC), Best Short Paper Award , Austin, Texas, Dec 2016

Bib

@inproceedings{schermann:16,
  author = {Schermann, Gerald and Cito, Jürgen and Leitner, Philipp and Gall, Harald C.},
  title = {Towards Quality Gates in Continuous Delivery and Deployment},
  booktitle = {Proceedings of the 24TH IEEE International Conference on Program Comprehension (ICPC), <strong>Best Short Paper Award</strong>},
  year = {2016},
  location = {Austin, Texas},
  pages = {nn}
}

TOIT

Patterns in the Chaos - A Study of Performance Variation and Predictability in Public IaaS Clouds

Philipp Leitner, and Jürgen Cito

ACM Transactions on Internet Technology, Apr 2016

Bib PDF

@article{leitner:16,
  author = {Leitner, Philipp and Cito, Jürgen},
  title = {Patterns in the Chaos - A Study of Performance Variation and Predictability in Public IaaS Clouds},
  journal = {ACM Transactions on Internet Technology},
  issue_date = {June 2016},
  volume = {16},
  number = {3},
  month = apr,
  year = {2016},
  issn = {1533-5399},
  pages = {15:1--15:23},
  articleno = {15},
  numpages = {23},
  doi = {10.1145/2885497},
  acmid = {2885497},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {Infrastructure-as-a-service, benchmarking, public cloud},
}

2015

All the Services Large and Micro: Revisiting Industrial Practice in Services Computing

Gerald Schermann, Jürgen Cito, and Philipp Leitner

In Proceedings of the 11th International Workshop on Engineering Service Oriented Applications (WESOA’15) , Goa, India, Apr 2015

Bib

@inproceedings{scherman:15:1,
  author = {Schermann, Gerald and Cito, Jürgen and Leitner, Philipp},
  title = {All the Services Large and Micro: Revisiting Industrial Practice in Services Computing},
  booktitle = {Proceedings of the 11th International Workshop on Engineering Service Oriented Applications (WESOA'15)},
  year = {2015},
  pages = {nn},
  location = {Goa, India}
}

A Framework for a Cost-Efficient Cloud Ecosystem

Eliot Salant, Philipp Leitner, Karl Wallbom, and James Athes

In eChallenges e-2015 Conference , Vilnius, Lithuania, Apr 2015

Bib

@inproceedings{salant:15,
  author = {Salant, Eliot and Leitner, Philipp and Wallbom, Karl and Athes, James},
  title = {{A Framework for a Cost-Efficient Cloud Ecosystem}},
  booktitle = {{eChallenges e-2015 Conference}},
  year = {2015},
  pages = {nn},
  location = {Vilnius, Lithuania}
}

Bursting With Possibilities – an Empirical Study of Credit-Based Bursting Cloud Instance Types

Philipp Leitner, and Joel Scheuner

In Proceedings of the 8th IEEE/ACM International Conference on Utility and Cloud Computing (UCC) , Limassol, Cyprus, Apr 2015

Bib

@inproceedings{leitner:15,
  author = {Leitner, Philipp and Scheuner, Joel},
  title = {Bursting With Possibilities – an Empirical Study of Credit-Based Bursting Cloud Instance Types},
  booktitle = {Proceedings of the 8th IEEE/ACM International Conference on Utility and Cloud Computing (UCC)},
  year = {2015},
  location = {Limassol, Cyprus},
  pages = {nn}
}

Intent, Tests, and Release Dependencies: Pragmatic Recipes for Source Code Integration

Martin Brandtner, Philipp Leitner, and Harald C. Gall

In Proceedings of the 15th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) , Bremen, Germany, Apr 2015

Bib

@inproceedings{brandtner:15:2,
  author = {Brandtner, Martin and Leitner, Philipp and Gall, Harald C.},
  title = {Intent, Tests, and Release Dependencies: Pragmatic Recipes for Source Code Integration},
  booktitle = {Proceedings of the 15th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM)},
  year = {2015},
  location = {Bremen, Germany},
  pages = {nn}
}

CloudWave - Leveraging DevOps for Cloud Management and Application Development

Dario Bruneo, Aryan Dadashi, Philipp Leitner, Boris Moltchanov, Francesco Javier Nieto De-Santos, Avi Miron, Eliot Salant, Amir Molzam Sharifloo, Karl Wallbom, and Chris Woods

In Proceedings of the Fourth European Conference on Service-Oriented and Cloud Computing (ESOCC’15) - EU Projects Track , Taormina, Italy, Apr 2015

Bib

@inproceedings{bruneo:15,
  author = {Bruneo, Dario and Dadashi, Aryan and Leitner, Philipp and Moltchanov, Boris and De-Santos, Francesco Javier Nieto and Miron, Avi and Salant, Eliot and Sharifloo, Amir Molzam and Wallbom, Karl and Woods, Chris},
  title = {CloudWave - Leveraging DevOps for Cloud Management and Application Development},
  booktitle = {Proceedings of the Fourth European Conference on Service-Oriented and Cloud Computing (ESOCC'15) - EU Projects Track},
  year = {2015},
  location = {Taormina, Italy},
  pages = {nn}
}

JCloudScale: Closing the Gap Between IaaS and PaaS

Rostyslav Zabolotnyi, Philipp Leitner, Waldemar Hummer, and Schahram Dustdar

ACM Transactions on Internet Technology, Jul 2015

Bib

@article{zabolotnyi:15,
  author = {Zabolotnyi, Rostyslav and Leitner, Philipp and Hummer, Waldemar and Dustdar, Schahram},
  title = {JCloudScale: Closing the Gap Between IaaS and PaaS},
  journal = {ACM Transactions on Internet Technology},
  issue_date = {September 2015},
  volume = {15},
  number = {3},
  month = jul,
  year = {2015},
  issn = {1533-5399},
  pages = {10:1--10:20},
  articleno = {10},
  numpages = {20},
  doi = {10.1145/2792980},
  acmid = {2792980},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {Cloud computing, JCloudScale, middleware, programming}
}

The Making of Cloud Applications – An Empirical Study on Software Development for the Cloud

Jürgen Cito, Philipp Leitner, Thomas Fritz, and Harald C. Gall

In Proceedings of the 10th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE) , Bergamo, Italy, Jul 2015

Bib

@inproceedings{cito:15:3,
  author = {Cito, Jürgen and Leitner, Philipp and Fritz, Thomas and Gall, Harald C.},
  title = {The Making of Cloud Applications – An Empirical Study on Software Development for the Cloud},
  booktitle = {Proceedings of the 10th Joint Meeting of the European Software Engineering
                 Conference and the {ACM} {SIGSOFT} International Symposium on Foundations
                 of Software Engineering (ESEC/FSE)},
  year = {2015},
  location = {Bergamo, Italy},
  publisher = {ACM},
  pages = {393--403},
  address = {New York, NY, USA}
}

Onward! @ SPLASH 2015

Runtime Metric Meets Developer - Building Better Cloud Applications Using Feedback

Jürgen Cito, Philipp Leitner, Harald C. Gall, Aryan Dadashi, Anne Keller, and Andreas Roth

In Proceedings of the 2015 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software (Onward! 2015) , Pittsburgh, Pennsylvania, United States, Jul 2015

Bib PDF

@inproceedings{cito:15:2,
  author = {Cito, Jürgen and Leitner, Philipp and Gall, Harald C. and Dadashi, Aryan and Keller, Anne and Roth, Andreas},
  title = {Runtime Metric Meets Developer - Building Better Cloud
               Applications Using Feedback},
  booktitle = {Proceedings of the 2015 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming \& Software (Onward! 2015)},
  year = {2015},
  pages = {14--27},
  location = {Pittsburgh, Pennsylvania, United States},
  publisher = {ACM},
  address = {New York, NY, USA},
}

Identifying Web Performance Degradations Through Synthetic and Real-user Monitoring

Jürgen Cito, Devan Gotowka, Philipp Leitner, Ryan Pelette, Dritan Suljoti, and Schahram Dustdar

Journal of Web Engineering, Nov 2015

Bib

@article{cito:15,
  author = {Cito, Jürgen and Gotowka, Devan and Leitner, Philipp and Pelette, Ryan and Suljoti, Dritan and Dustdar, Schahram},
  title = {Identifying Web Performance Degradations Through Synthetic and Real-user Monitoring},
  journal = {Journal of Web Engineering},
  issue_date = {November 2015},
  volume = {14},
  number = {5-6},
  month = nov,
  year = {2015},
  issn = {1540-9589},
  pages = {414--442},
  numpages = {29},
  acmid = {2871278},
  publisher = {Rinton Press, Incorporated},
  address = {Paramus, NJ},
  keywords = {application performance monitoring, changepoint analysis, performance engineering, web services}
}

SPEEDL - A Declarative Event-Based Language for Cloud Scaling Definition

Rostyslav Zabolotnyi, Philipp Leitner, Stefan Schulte, and Schahram Dustdar

In Proceedings of the Future of Software Engineering For and In Cloud, Visionary Track of IEEE Services , Nov 2015

Bib

@inproceedings{zabolotnyi:16,
  author = {Zabolotnyi, Rostyslav and Leitner, Philipp and Schulte, Stefan and Dustdar, Schahram},
  title = {SPEEDL - A Declarative Event-Based Language for Cloud Scaling Definition},
  booktitle = {Proceedings of the Future of Software Engineering For and In Cloud, Visionary Track of IEEE Services},
  year = {2015},
  pages = {nn}
}

Profile-based View Composition in Development Dashboards

Martin Brandtner, Philipp Leitner, and Harald Gall

In Proceedings of the 17th Workshop Software-Reengineering & Evolution (WSRE) , Nov 2015

Bib

@inproceedings{brandtner:15,
  author = {Brandtner, Martin and Leitner, Philipp and Gall, Harald},
  booktitle = {Proceedings of the 17th Workshop Software-Reengineering & Evolution (WSRE)},
  title = {Profile-based View Composition in Development Dashboards},
  year = {2015},
  pages = {nn}
}

Discovering Loners and Phantoms in Commit and Issue Data

Gerald Schermann, Martin Brandtner, Sebastiano Panichella, Philipp Leitner, and Harald C. Gall

In Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension , Florence, Italy, Nov 2015

Bib

@inproceedings{schermann:15,
  author = {Schermann, Gerald and Brandtner, Martin and Panichella, Sebastiano and Leitner, Philipp and Gall, Harald C.},
  title = {Discovering Loners and Phantoms in Commit and Issue Data},
  booktitle = {Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension},
  series = {ICPC '15},
  year = {2015},
  location = {Florence, Italy},
  pages = {4--14},
  numpages = {11},
  acmid = {2820287},
  publisher = {IEEE Press},
  address = {Piscataway, NJ, USA}
}

Cloud WorkBench: Benchmarking IaaS Providers Based on Infrastructure-as-Code

Joel Scheuner, Jürgen Cito, Philipp Leitner, and Harald C. Gall

In Proceedings of the 24th International Conference on World Wide Web , Florence, Italy, Nov 2015

Bib

@inproceedings{scheuner:15,
  author = {Scheuner, Joel and Cito, Jürgen and Leitner, Philipp and Gall, Harald C.},
  title = {Cloud WorkBench: Benchmarking IaaS Providers Based on Infrastructure-as-Code},
  booktitle = {Proceedings of the 24th International Conference on World Wide Web},
  series = {WWW '15 Companion},
  year = {2015},
  isbn = {978-1-4503-3473-0},
  location = {Florence, Italy},
  pages = {239--242},
  numpages = {4},
  doi = {10.1145/2740908.2742833},
  acmid = {2742833},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {benchmarking, cloud computing, devops, iaas, iac}
}

SQA-Profiles: Rule-Based Activity Profiles for Continuous Integration Environments

Martin Brandtner, Sebastian C. Müller, Philipp Leitner, and Harald Gall

In Proceedings of the 22nd IEEE International Conference on Software Analysis, Evolution, and Reengineering (SANER’15) , Nov 2015

Bib

@inproceedings{brandtner:15:3,
  author = {Brandtner, Martin and Müller, Sebastian C. and Leitner, Philipp and Gall, Harald},
  booktitle = {Proceedings of the 22nd IEEE International Conference on Software Analysis, Evolution, and Reengineering (SANER'15)},
  title = {SQA-Profiles: Rule-Based Activity Profiles for Continuous Integration Environments},
  year = {2015},
  pages = {nn}
}

Comparing and Combining Predictive Business Process Monitoring Techniques

Andreas Metzger, Philipp Leitner, Dragan Ivanovic, Eric Schmieders, Rod Franklin, Manuel Carro, Schahram Dustdar, and Klaus Pohl

IEEE Transactions on Systems, Man, and Cybernetics: Systems, Feb 2015

Bib

@article{metzger:14,
  author = {Metzger, Andreas and Leitner, Philipp and Ivanovic, Dragan and Schmieders, Eric and Franklin, Rod and Carro, Manuel and Dustdar, Schahram and Pohl, Klaus},
  journal = {IEEE Transactions on Systems, Man, and Cybernetics: Systems},
  title = {Comparing and Combining Predictive Business Process Monitoring Techniques},
  year = {2015},
  volume = {45},
  number = {2},
  pages = {276-290},
  keywords = {business data processing;constraint satisfaction problems;learning (artificial intelligence);QoS aggregation;accuracy indicators;constraint satisfaction;machine learning;predictive business process monitoring techniques;process execution;quality-of-service aggregation;Accuracy;Business;Data models;Monitoring;Predictive models;Quality of service;Real-time systems;Business data processing;failure analysis;forecasting;neural network applications;transportation},
  doi = {10.1109/TSMC.2014.2347265},
  issn = {2168-2216},
  month = feb,
  top = {true}
}

2014

CloudCom 2014

Cloud WorkBench - Infrastructure-as-Code Based Cloud Benchmarking

Joel Scheuner, Philipp Leitner, Jürgen Cito, and Harald Gall

In Proceedings of the 6th IEEE International Conference on Cloud Computing Technology and Science (CloudCom’14) , Feb 2014

Bib PDF

@inproceedings{scheuner:14,
  author = {Scheuner, Joel and Leitner, Philipp and Cito, Jürgen and Gall, Harald},
  booktitle = {Proceedings of the 6th IEEE International Conference on Cloud Computing Technology and Science (CloudCom'14)},
  title = {Cloud WorkBench - Infrastructure-as-Code Based Cloud Benchmarking},
  year = {2014},
  pages = {nn},
  doi = {10.1109/CloudCom.2014.98},
}

WPress: Benchmarking Infrastructure-as-a-Service Cloud Computing Systems for On-line Transaction Processing Applications

Amir Hossein Borhani, Philipp Leitner, Bu-Sung Lee, Xiaorong Li, and Terence Hung

In Proceedings of the 18th IEEE International Enterprise Distributed Object Computing Conference (EDOC) , Feb 2014

Bib

@inproceedings{borhani:14,
  author = {Borhani, Amir Hossein and Leitner, Philipp and Lee, Bu-Sung and Li, Xiaorong and Hung, Terence},
  booktitle = {Proceedings of the 18th IEEE International Enterprise Distributed Object Computing Conference (EDOC)},
  title = {WPress: Benchmarking Infrastructure-as-a-Service Cloud Computing Systems for On-line Transaction Processing Applications},
  year = {2014},
  pages = {nn}
}

Profiling-Based Task Scheduling for Factory-Worker Applications in Infrastructure-as-a-Service Clouds

Rostyslav Zabolotnyi, Philipp Leitner, and Schahram Dustdar

In Proceedings of the 40th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) , Feb 2014

Bib

@inproceedings{zabolotnyi:14:2,
  author = {Zabolotnyi, Rostyslav and Leitner, Philipp and Dustdar, Schahram},
  booktitle = {Proceedings of the 40th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)},
  title = {Profiling-Based Task Scheduling for Factory-Worker Applications in Infrastructure-as-a-Service Clouds},
  year = {2014},
  pages = {nn}
}

CloudWave: where Adaptive Cloud Management Meets DevOps

Dario Bruneo, Thomas Fritz, Sharon Keidar-Barner, Philipp Leitner, Francesco Longo, Clarissa Marquezan, Andreas Metzger, Klaus Pohl, Antonio Puliafito, Danny Raz, Andreas Roth, Eliot Salant, Itai Segall, Massimo Villari, Yaron Wolfsthal, and Chris Woods

In Proceedings of the Fourth International Workshop on Management of Cloud Systems (MoCS 2014) , Feb 2014

Bib

@inproceedings{bruneo:14,
  title = {CloudWave: where Adaptive Cloud Management Meets DevOps},
  author = {Bruneo, Dario and Fritz, Thomas and Keidar-Barner, Sharon and Leitner, Philipp and Longo, Francesco and Marquezan, Clarissa and Metzger, Andreas and Pohl, Klaus and Puliafito, Antonio and Raz, Danny and Roth, Andreas and Salant, Eliot and Segall, Itai and Villari, Massimo and Wolfsthal, Yaron and Woods, Chris},
  booktitle = {Proceedings of the Fourth International Workshop on Management of Cloud Systems (MoCS 2014)},
  year = {2014},
  pages = {nn}
}

Identifying Root Causes of Web Performance Degradation Using Changepoint Analysis

Jürgen Cito, Dritan Suljoti, Philipp Leitner, and Schahram Dustdar

In Web Engineering: 14th International Conference, ICWE 2014, Toulouse, France, July 1-4, 2014. Proceedings , Feb 2014

Bib

@inproceedings{cito:14,
  author = {Cito, Jürgen and Suljoti, Dritan and Leitner, Philipp and Dustdar, Schahram},
  editor = {Casteleyn, Sven and Rossi, Gustavo and Winckler, Marco},
  title = {Identifying Root Causes of Web Performance Degradation Using Changepoint Analysis},
  booktitle = {Web Engineering: 14th International Conference, ICWE 2014, Toulouse, France, July 1-4, 2014. Proceedings},
  year = {2014},
  publisher = {Springer International Publishing},
  address = {Cham},
  pages = {181--199},
  isbn = {978-3-319-08245-5},
  doi = {10.1007/978-3-319-08245-5_11},
}

A Note on Software Tools and Techniques for Monitoring and Prediction of Cloud Services

Rajiv Ranjan, Rajkumar Buyya, Philipp Leitner, Armin Haller, and Stefan Tai

Software: Practice and Experience, Jul 2014

Bib

@article{ranjan:14,
  author = {Ranjan, Rajiv and Buyya, Rajkumar and Leitner, Philipp and Haller, Armin and Tai, Stefan},
  title = {A Note on Software Tools and Techniques for Monitoring and Prediction of Cloud Services},
  journal = {Software: Practice and Experience},
  issue_date = {July 2014},
  volume = {44},
  number = {7},
  month = jul,
  year = {2014},
  issn = {0038-0644},
  pages = {771--775},
  numpages = {5},
  doi = {10.1002/spe.2266},
  acmid = {2904546},
  publisher = {John Wiley \& Sons, Inc.},
  address = {New York, NY, USA}
}