Overcoming DevSecOps Challenges for Resilient Solutions

Author: Julia Kanaikina

DevSecOps is a seamless blend of software development, security, and operations, designed to integrate these different realms into a harmonious cycle of continuous delivery. However, the process of integrating security into DevOps is unique for each case and comes with specific obstacles. In this article, we aim to share IBA Group’s experience gained during the implementation of a DevSecOps solution in one of our projects. We will shed light on how we managed to overcome significant barriers and successfully implement an effective DevSecOps approach. This article is part of IBA Group’s DevSecOps series, and if you are interested in delving deeper into the topic, you can read the first article and the second article in our DevSecOps series.

Challenges of SAST Integration into the CI/CD Pipeline

Static Application Security Testing (SAST) integration plays a vital role in the Software Development Life Cycle (SDLC), and it is a fundamental component of CI/CD security as it detects significant vulnerabilities in an application prior to deployment to production, when the remediation costs of vulnerabilities are comparatively low. Based on the customer requirements, we chose SonarQube as a SAST tool, a renowned platform for continuous inspection of code quality. SonarQube has a great capacity to detect bugs and security vulnerabilities. However, SAST integration into the CI/CD pipeline exposed us to an array of challenges hidden before.

Addressing Vulnerabilities and Bugs Discovered by SonarQube

The introduction of SonarQube led the project management team to an unexpected revelation – a massive volume of vulnerabilities and bugs hidden within the project. The team found themselves at a standstill, as fixing these issues impeded further development and caused significant time delays. At that point, continuing product development had a higher priority than stopping and fixing vulnerabilities in the code. Moreover, project management did not want to be completely blocked by found vulnerabilities in the code. To address this, we proposed a quick and short-term solution: we decided to run SonarQube without failing the pipeline automatically so that it does not block the build, but the team is aware of vulnerabilities.

Flexible Approach and Incremental Vulnerability Resolution

As the second step, our DevSecOps team developed a more flexible solution. Recognizing the crucial security role and the necessity to maintain the speed of product development, we developed a plan to address the issues highlighted by SonarQube incrementally. We prioritized vulnerabilities based on their risk factor and impact on the project, fixing them in controlled batches. This allowed the team to keep development progressing while steadily reducing the threat landscape. Alongside, we conducted security coding training for the development team and provided rigorous manual code reviews to catch potential bugs before they became ingrained within the codebase.

Integration of Dynamic Application Security Testing (DAST)

Another type of testing that our team integrated into the customer’s software development lifecycle (SDLC) was Dynamic Application Security Testing (DAST) tool. DAST integration is crucial for ensuring the security and robustness of applications. The DAST tool is aimed at testing the application during the testing or deployment phase to identify vulnerabilities and weaknesses that may only occur during runtime. Initially, we integrated it only in the staging environment. At the same time, we faced a problem: the CI/CD pipeline slowed down by three times, which was unacceptable for the team. Thus, our goal was to achieve CI/CD with integrated security without compromising the speed.

Balancing Speed and Security in the CI/CD Pipeline

To achieve this goal, we decided to define the scope of the application that should be tested in the first place. Through a combination of code analysis, threat modeling, and vulnerability assessment, we managed to identify and prioritize the areas that demanded the utmost level of attention and could consequently lead to security breaches. The DAST tool was integrated into the CI/CD pipeline and configured to test only these specific parts of the application, so we were able to speed up the testing process. Furthermore, we developed other configurations for the DAST tool to cover other types of vulnerabilities. As a result, they could be run upon request or scheduled on a weekly basis, for instance, outside the CI/CD pipeline.

Fostering Organizational Awareness and Adaptability

However, dealing with technical issues was only part of the solution. The key to successful Security integration into DevOps lies in raising the awareness of business management. Security is not merely a technical concern but an organizational one. We held awareness sessions explaining the role of DevSecOps, its benefits, and the potential risks of not using it. Our efforts led to a better understanding of DevSecOps amongst business leaders causing their active involvement and support.

Embracing a New Working Model and Cultivating Change

SAST and DAST integrations into the CI/CD pipeline episode was just one facet of the multifaceted challenge we faced. Another significant hurdle that reared its head was the difficulty in adapting to a new dynamic working model. The key of DevSecOps lies in its ability to foster rapid, iterative development while ensuring continuous security integration. This represents a significant shift from traditional phase-based software development methods. As a result, our team found itself grappling with the challenges of changing gears midway through the project.

Customer’s team was used to designating phases for development, security, and operations but DevSecOps made us blur these lines. Unexpectedly, everyone had to wear multiple hats – developers had to think about security, and the operations team had to get involved in the early stages of development. This required not just a change in mindset, but also a shift in our practical approach.

However, the solution was not to return to our comfort zones, but in embracing the change and pushing our boundaries. We initiated comprehensive training programs for the team to better understand their new roles. Knowledge-sharing sessions were held where different teams could learn from each other. We turned to fostering a culture that welcomed change and encouraged continuous learning.

Initially, it was a struggle. But gradually, the team started experiencing the benefits of this integrated approach. Improved communication between the teams led to a better understanding and collaboration. Constant feedback cycle helped us identify and rectify problems much earlier in the development phase. As the result, security integration helped to promote a culture focused on continuous improvement, where security practices are regularly evaluated, updated, and adapted to emerging threats. With time, what seemed like a steep mountain became an easily navigable hill.

Growing as a Team and Achieving Organizational Resilience

This shared journey has not only enabled us to overcome the challenges we faced but has also facilitated the growth of our team and organization, enhancing our problem-solving abilities and fostering organizational resilience.

At IBA Group, we firmly believe that every challenge presents an opportunity for growth and learning. If you are seeking to implement DevSecOps into your development cycle, we are here to provide assistance. Our expert team, backed by extensive experience, is prepared to address any inquiries, challenges, or requirements that may arise on your DevSecOps journey. Let us combine our efforts and collaborate in creating secure and resilient software solutions together.

Keep in mind, the key lies not in finding a path without obstacles, but in harnessing the strength and expertise to navigate through them. This is the commitment that IBA Group guarantees to deliver.