-
What is the Source Code Scanner?
The Source Code Scanner is a static scanning tool that uses Checkmarx security technology (hence it is often just referred to as the Checkmarx Scanner). It is mandatory for any security review submission that includes a Salesforce package or component. The Scanner checks Apex, Visualforce, and Lightning code, but doesn't check external endpoints of a solution, nor does it work on mobile clients or for API solutions.
-
How many scans can I run?
Partners scanning for the
AppExchange Security Review can use 3 free scans per review. Consider running an alternative tool as you develop, such as
Salesforce Code Analyzer, or the
PMD Extension for VS Code, and the Source Code Scanner as you finalize your submission. If you want the flexibility and freedom to scan unpackaged code, or bypass scan limits and package linking requirements, you can also purchase a license from Checkmarx directly, but this is typically more financially feasible for large organizations.
-
How do I run a Checkmarx scan?
To successfully run Checkmarx via this portal, you must submit scans via a user with Author Apex accounts (e.g. a System Admin user) associated to the packaging organization (this would either be a Developer Edition org where you're packaging your code directly, or if you're using a Dev Hub org with SFDX packaging methodology, you would use your Dev Hub org) for your Appexchange package. This packaging organization
must be added to your Partner account on the AppExchange.
-
The package version I want to scan is not showing up in the Portal. What can I do?
The scanner will only show packages meeting the following criteria:
- The package must be managed-beta or managed-released. In the case of 2GP, this means you need to “promote” the package version. You will only see package versions that are also showing up in the Partner Community → Publishing → Technologies —> Solutions tab.
- You need to be using the same Dev Hub or Developer Edition org that you used to create the package you want to scan.
- Only the last 10 versions of any given package will show up in the portal.
-
Does the scanner work for both 1GP and 2GP packages?
Yes, the scanner has been successfully tested on both 1GP and 2GP packages.
- For 1GP packages, you will need to access the scanner using the Developer Edition org where you developed the package. Make sure the package you want to scan has been uploaded as “Managed - Released” not beta, as is selected by default when you upload a package.
- For 2GP packages, you'll need to use the Dev Hub org you created the packages with to access the Portal. Make sure the package version you want to scan has been promoted to 'Released'.
-
My package doesn't contain any Apex, Visualforce, or Lightning Code. Do I still need to scan it?
If your package was created declaratively without adding custom code, there may not be any scannable content in the package. For example, if your package contains only things like custom objects, fields, reports, flow templates, dashboards, etc. you might not get any results at all from Checkmarx. In this case, for your security review submission, in place of Checkmarx results, you can upload a document stating that your package doesn't contain any code so the Checkmarx scan requirement is not applicable.
-
My Checkmarx results were clean, but my security review still failed. Is the Security Review Team using a different scanning tool?
Those familiar with static analysis know that there is no single tool that can find all bugs.
It's also well understood that these tools, because they lack insight into the context of the application, can produce false positives as well. It is important to recognize that false negatives and false positives exist in these reports and any given report should not be considered a full and outright security assessment of your application and code. Manual reviews will always be necessary to verify code correctness. That being said, besides manual code review, the Product Security Team also makes use of Salesforce Code Analyzer, and/or the PMD Extension for VS Code, which we would strongly encourage you to use too. Using one of these tools along with Checkmarx will maximize your chances of finding all the Salesforce-code-specific bugs like CRUD/FLS violations, Sharing violations, SOQL injection, etc.
-
What is Chimera?
Chimera is a cloud-based security scanning service that combines 4 open-source security scanning tools into one service. Chimera is powered by and scans from the Heroku cloud platform. Custom-built code then combines and analyzes the results from all scanning tools used and provides a single, actionable security report to you.
You would use this scanner on any external (non-Salesforce) integrations that you own/develop and are planning to connect to an AppExchange offering. You would not use this on Salesforce package code or Salesforce web endpoints or APIs.
If you are interested in doing so, you may download any of the component scanners that make up Chimera as stand-alone tools and run them against your projects internally. Downloading and running tools individually may be required for mobile or client applications, as well as web applications not accessible to the public internet.
The open source scanners currently run as part of Chimera are:
- OWASP Zed Attack Proxy (ZAP): OWASP's Zed Attack Proxy (ZAP) is an open-source penetration testing, web scanning, and attack proxy tool. By making automated requests to your web application ZAP conducts automated black box testing of common web vulnerabilities such as XSS, SQL Injection, CSRF, and many others.
- Nmap: Nmap is a utility for network discovery, allowing us to determine which ports are currently open and accepting connections on your server. This helps an attacker identify which services are installed on your server and potentially which may be vulnerable for exploitation.
- Nikto2: Nikto is a web server scanner which performs black box tests against servers for common vulnerabilities and informational items. This includes vulnerable versions of software which may be installed, dangerous files or programs on the server, and default server or application pages that may still be publicly accessible due to misconfiguration or oversight.
- SSLyze: SSLyze is a scanner that analyzes the SSL configuration and certificate of your server. Through use of SSLyze and other, internal tools, Chimera will report on any misconfigurations or outdated/weak ciphers and libraries your server may be at risk by using.
Note: while OWASP ZAP scan results are an acceptable substitute for Chimera for an AppExchange Security Review submission, none of these other 3 scanning tools (Nmap, Nikto2, or SSLyze) are by themselves a sufficient substitute.
-
Does Chimera work on all websites, APIs, and services?
Chimera will not work on every site. Please note the following requirements:
- Chimera is only able to scan web applications that are reachable from the public internet.
- You must own the server hosting the site or service, because we require you to add an abuse prevention token file to the root of your server in order to run the scanner.
- Chimera won't work on AWS-hosted apps.
- Chimera relies on having a lot of "web surface area" to scan, so it tends not to be as useful for API-only systems.
- Chimera won't work for mobile or client apps that aren't reachable by public Internet, so please instead use OWASP ZAP for these types of apps as per instructions put together here.
-
How do I run Chimera?
- Click the 'Create New Chimera Scan' button.
- A window will pop up with a link to an abuse prevention token and a form to configure your scan request.
- Download the token file and upload it as a .txt file to the root of the server you want to test. You should be able to access the file at yourwebsite.com/tokenfilename.txt (if you can't access it, Chimera won't be able to either)
- Once this is done, fill out the other fields in the popup form:
- Target: the base URL of the site/service you want to scan
- Do Not Scan: URLs that you want the scanner to avoid
- Testing Username: username for the scanner to login with
- Testing Password: password for username above
- Once you submit the scan, you'll be taken back to the Portal dashboard where you can view the status of the scan.
- Scans can take a long time. The status should start as "Queued", move to "Working", move to "Ready to Generate", and once done, should show as "Completed." You should receive the results by email a while after the scan completes.
-
Should I use Chimera on my Production service?
While Chimera will run correctly against your Production service, we recommend you provide a representative staging or development environment so as to mitigate the risk of data loss or corruption. The environment provided to Chimera must be accessible from the public internet.
-
How long does a scan usually take?
Scan times vary widely due to factors such as volume, queue size, application size, and network speed/connectivity of the application servers. In general, we see scans complete anywhere from 4-16 hours after starting. Time in queue varies greatly and we cannot provide estimates at this time.
-
Can you give me the IP address/range that you will scan from?
Due to the cloud-based nature of Chimera (we use Heroku for scalability), we are unable to give you an IP address or range that the scanner engines will use. At this time we have no plans to convert to a static IP address or range for Chimera to use. If we do implement this, we will update this page with the IP ranges you can expect to see traffic from.
-
My Chimera report says it was unable to login, but I provided test credentials ?
Chimera does its best to identify how to login to your web application automatically, but this service is not perfect. We are constantly working on improving this service and hope to have a significantly more powerful auto-login engine online soon. We log every failed attempt to determine how to login for jobs that credentials were provided for and investigate to learn from it, so there is no need to email us about every failure. In cases where Chimera was unable to login to your site automatically, we recommend that you
download ZAP and run a security scan locally. We will periodically post announcements on this page relating to new releases, including new auto-login engine releases.
-
My scan is hung or stuck. What can I do?
Troubleshooting steps:
- You may need to wait a day for the scan to run, depending on the scanner bandwidth and “size” of your scan.
- Double-check that your abuse prevention token can be accessed at yourwebsite.com/tokenfilename.txt
- Check your email to see if you received any error notifications about the scan, or if the results were sent to spam.
- Check your website logs to make sure the scanner was not blocked from your side.
- Double-check the criteria listed in the question above "Does Chimera work on all websites, APIs, and services?"
- If you've waited 48 hours with no results, you may want to try re-initiating the scan.
-
I've tried everything and can't get Chimera to work. What should I do?
Since Chimera is not well-supported, you won't be able to get much assistance by logging a Partner Community case. If you can't get the scanner to work, we'd recommend running either just (1) OWASP ZAP, and additionally, if you'd like, any of the other tools that make up the Chimera scanner individually: (2) NMap, (3) Nikto2, and (4) SSLyze. The latter 3 are optional for your security review submission (we will accept OWASP ZAP scan results in place of Chimera, but the other 3 tools would not be valid substitutes on their own because they have narrower scopes). If you have a license for the Burp Suite Scanner (available from PortSwigger), we will also gladly accept Burp scan results in place of Chimera or OWASP ZAP.
-
Can I email you questions about how to fix my security issues?
Due to the volume of security scans provided by Chimera, we cannot answer technical security questions (which tend to be quite in-depth discussions) on an ad-hoc basis. If you have questions about the issues found on your site, please schedule an Office Hour appointment with the Salesforce Product Security team via this same Portal.
-
Do I have to fix every issue on the report?
Chimera reports endeavor to give you all information about all issues or potential issues discovered on your site with as much documentation as it is possible for us to provide. Some of these issues may be false positives, or may not be valid security issues. We ask that you carefully review the report and provided documentation and reference. If you have any questions, we are happy to help. Please schedule an Office Hour appointment with the Salesforce Product Security team.
-
Can I use Chimera if I am not an ISV Partner?
At this time, Chimera is only available to ISV Partners developing external integrations for the AppExchange.
-
Despite having clean Chimera results, my application still failed security review. Is the Security Review Team using a different scanning tool?
The Security Review is a manual process, performed by one of our security engineers. Having clean Chimera scan results does not guarantee that your external endpoints are free of security issues, nor should you rely on any single tool for this.
All tools (as well as human reviewers) have limited coverage, therefore your software development methodology should include a variety of different security checks, so that issues that are missed by one check are caught by another. For example, architectural reviews during the design phase, linters used as pre-commit hooks, periodic code reviews to ensure compliance with our Secure Coding Guidelines, static analysis scans with other tools like Salesforce Code Analyzer, dynamic analysis testing using webdriver or a similar tool for websites, use of API fuzzers, and even third party penetration testing or code audits are all part of a modern software development process.
That being said, one of the best tools for conducting your own security analysis of non-Salesforce endpoints is Burp Suite. At this time, Salesforce cannot provide licenses for this tool, but many features are also available with a free license. Learn more about testing with Burp here.