Feedback requested: New security configurations feature! 🎉

[KateCatlin]

What's new?

We're excited to announce the launch of our latest feature, security configurations! We're eager to hear what you think!

With this feature, you can:

• Easily create a modular set of security settings for groups of repositories.
• Explore and implement our GitHub recommended security configurations as a starting point.
• Establish a default configuration for all new repositories with essential security features enabled.
• See the additional GHAS licenses required to apply a configuration (or freed up by disabling GHAS).

What's next?

This release marks the beginning of our public beta phase!

There's still more to come before we officially roll out the feature. Now is the perfect time for your feedback to help us shape its future 🎉 🚀

Let's talk!

Got questions, comments, concerns, or just want to share some kudos? We're all ears! Your input matters, so let's start a conversation and make this feature even better together.

[makskustomer]

This is a great way to enforce this without getting code involved. Regarding configs that fail to apply; it would be helpful to understand why they failed and whether an old codeql config is present that needs to be removed. If so, help us remove the old configs en mass.

[makskustomer]

Couple of other issues surfaced:

1. When filtering by config-status:failed on the configurations page, it also shows repos without a config applied or that may have failed in the past but no longer fail.
2. In global settings and configurations, there is no way to define what is part of the default or extended setup. This is particularly troublesome if you want to include languages not part of the default but have no way to do so.
3. The above as well as a quick way remove old configs is causing about 2/3 of my repos to fail without manual intervention on my part.

[KateCatlin]

Hi @makskustomer thank you so much for the excellent feedback! I'm documenting all of these and will circle back when they are addressed. Appreciate you jumping in on the beta so fast.

[KateCatlin]

Quick follow-up here @makskustomer - I'd love to jump on a call and learn more about your use case on a couple of these. Do you have ~25 minutes to chat? If so, you can book me here!

[djredman99]

+1 on the need to visually provide customers a means to understand why configurations have failed to apply. Experienced this first-hand while working with a customer.

[unchris]

We've been working on a script to automate this recently - glad to see it's being baked into the UI, it's definitely going to save us some time for the basic Enable/Disable toggles!

Some feedback:

1. Our company uses enterprise cloud with EMU and lots of organizations. It would be nice if this could be set at the enterprise level instead of having to configure this per-org.
2. For configuring CodeQL or any other tool that has language-dependent features, it would be nice if the language detection feature in each repo could be combined with creating the boilerplate advanced CodeQL workflow that allows you to (among other things) set the language input to codeql-action/init
3. For repositories that have code compilation needs, it would be nice if when you apply a configuration, it could notify or easily dashboard which repos are failing after applying the CodeQL workflow.
4. This strays somewhat outside of security, but I assume Dependabot Version update schedules are at least somewhat linked with whatever triggers Dependabot Security PRs? If so, it would be nice if dependabot.yml files could be configured similarly, even if we have to supply some "default code" for things like private registry support. Currently I have the task of creating hundreds of these dependabot.yml files with private registry configuration for maven, npm, and python using CodeArtifact, and I suspect we'll be adding a few new ones. It would be amazing if this could also be configured at org- or enterprise-level with a configuration tool like the one you've released today.

[KateCatlin]

Hi @unchris! Thank you for all of this! It's great feedback and we appreciate you jumping in so quickly.

1. Our company uses enterprise cloud with EMU and lots of organizations. It would be nice if this could be set at the enterprise level instead of having to configure this per-org.

Enterprise level configurations are coming soon!

2. For configuring CodeQL or any other tool that has language-dependent features, it would be nice if the language detection feature in each repo could be combined with creating the boilerplate advanced CodeQL workflow that allows you to (among other things) set the language input to codeql-action/init

That is a really interesting idea that I will circle back with the team and ask for their thoughts on!

3. For repositories that have code compilation needs, it would be nice if when you apply a configuration, it could notify or easily dashboard which repos are failing after applying the CodeQL workflow.

We agree, and better failure messaging is coming soon!

4. This strays somewhat outside of security, but I assume Dependabot Version update schedules are at least somewhat linked with whatever triggers Dependabot Security PRs? If so, it would be nice if dependabot.yml files could be configured similarly, even if we have to supply some "default code" for things like private registry support. Currently I have the task of creating hundreds of these dependabot.yml files with private registry configuration for maven, npm, and python using CodeArtifact, and I suspect we'll be adding a few new ones. It would be amazing if this could also be configured at org- or enterprise-level with a configuration tool like the one you've released today.

This is an interesting idea as well, I'll chat with the Dependabot Updates folks and get their take!

[a-takamin]

Hello. Thank you for this nice feature!

I found that even when I apply the configuration that doesn't require the Advanced Security licenses, the message "5 GitHub Advanced Security licenses required" displays.
But the number should be 0, and after applying, the licenses won't really be used up. The number of available licenses did not change.

Could you please see if there is a bug?

---

configuration:

• GitHub Advanced Security features: Exclude
• Dependency graph: Enable
  • Dependabot: Enable
  • Vulnerable function calls: Disable
  • Security updates: Enable
• Code scanning: Disable
• Secret scanning: Disable
  • Push protection: Disable
• Private vulnerability reporting: Enable
• Policy: Public, private and internal

---

Thank you in advance.

[KateCatlin]

Hey @a-takamin thanks for reaching out! Can you share the name of the organization name where this is happening to you so we can look into it? It would also be helpful to know the configuration name and the repo names where it's errantly showing that you need licenses.

Thanks so much! We'll look into this straight away.

[a-takamin]

Hello, @KateCatlin
I'm sorry for the closed discussion, but could I share the name of the org. and the configuration in a new support ticket?
That is because they are private information.
Thank you.

[jamestran201]

@a-takamin Thank you for reporting this issue! We can reproduce it now, and will work on a fix for this.

[markgarrigan]

How do I apply the configuration I created?

[catulii]

Hi @markgarrigan, you can apply configurations in the repository table below! You can search/filter for specific repositories and once you make a selection, you will see an Apply configuration dropdown button. Once you click that, you can select the configuration you want to apply to the selected repositories.

Does that make sense?

[markgarrigan]

Ahhh... I see. Just a little feedback. Maybe I'm just a "dumb" ;) user but that is not super intuitive to me.

The instructions above that table say this....

Apply configurations
2761 GitHub Advanced Security licenses available, 20239 in use by XXXXXX.
Select repositories to view license consumption information.

The last line might be improved by saying...
Select repositories to view license consumption information and apply configurations.

[catulii]

@markgarrigan no such thing! That's why we're in beta, to find these kinks and make the experience more intuitive, so feedback like this is the best. We can definitely make that copy more explicit, thank you for your suggestion.

[markgarrigan]

What does No Security Features Enabled mean?

It appears to me that there are security features enabled.

[markgarrigan]

There seems to be an issue figuring out which repos have security features applied.

I have 31 pages of repos.

The repo table shows 25 repos on each page. The first page seems to correctly show which configuration is applied. However, every subsequent page shows...

[KateCatlin]

Hey @markgarrigan! Thanks so much for letting us know about this. It was indeed a bug, and we got a fix up for it today so let us know if it's still occurring for you!

[MarshallOfSound]

Heya @KateCatlin 👋

Great feature, much easier than my old approach of opening 100 repos in new tabs 😆 It's a bit scary to operate though because it's not clear what exactly will happen when I apply a given policy to a set of repos. Would be great if the confirmation prompt gave you an outline of the "diff" that was going to be applied (if any) when the policies get applied.

Otherwise I still have to go through each repo and see what the current setting for each thing is just to be sure there isn't a weird setting somewhere for a good reason 😅

[KateCatlin]

Hey @MarshallOfSound! Always great hearing from you. I love this idea! Let's jump on a call and discuss it - I'll reach out!

[archangelneo18]

Can we also add a way to add security features via global settings, like adding dependencies review workflows/actions? Another way is to add critical block actions in the global settings without creating new rules, etc.

[KateCatlin]

Thanks for the feedback @archangelneo18! This is really interesting and I want to hear more about your use case here. Would you be willing to book some time for us to chat and hear more? If so, you can grab any slot open on my calendar here!

[makskustomer]

Asking for the inverse here, it would be good if we could stuff some of the global settings into the configurations so that we can apply what we want using the powerful filtering across multiple repos. As an example, dependabot rules, using autofix or a number of other settings. @KateCatlin Currently Im in a situation where I want to setup automatic prs for certain severity levels of dependabot and it will apply to all my repos instead of ones I specify.

[KateCatlin]

Interesting feedback, thanks @makskustomer!

We're working on exploring which additional features to migrate over to configurations so the specificity helps!

[DTrejo]

Should these settings be under the Security tab? Instead of settings?

[KateCatlin]

Hey @DTrejo, tell me more about that! What makes you expect it there instead of the settings page?

[netomi]

This looks like an interesting feature indeed, however we ran into some issues after the beta feature was enabled in April by default for all GitHub organizations.

To give some context:

At the Eclipse Foundation we use a tool called otterdog (https://github.com/eclipse-csi/otterdog) to manage our numerous GitHub organizations with an infrastructure as code approach.

Now when the tool creates a new repo, it also sets code security related settings right away as needed.

With the new security configurations, the following sequence of events happens:

• a repo gets created by the tool
• the tool sets e.g. secret scanning setting according to the configuration
• the GitHub System sets the value of secret scanning according to the legacy setting of secret_scanning_enabled_for_new_repositories

An example of these events can be seen here:

The same applies for dependabot alerts and security updates as they follow the same principle.

While the new security configurations are a great tool if you mainly use the GitHub Web UI to configure your repos and organizations, if you rely on tools that do that for you via the rest or graphql api, you have to apply all kind of annoying workarounds. In this case we would need to wait for the security configuration to be applied and then change the security settings as needed, but I would like to avoid that ofc.

It would be amazing if there would be an option to completely disable security configurations such that nothing will be set after a repo has been created. Its fine if during the repo creation the security settings are immediately set to some default that could be determined from the legacy settings. This is basically the behavior up till March 2024. If there would be a way to ensure that this behavior is retained that would be of tremendous help for us.

[netomi]

Hi Kate,

I guess you refer to this option here at the bottom:

This is not for the legacy configuration, but rather the new security configuration. If you have no security configuration setup (like you seem to have by default if you have not setup anything yourself sofar) where no configuration applies for any repo, the legacy configuration seems be applied (which btw is only visible when you opt out of the beta). That got me confused for some time till I made the connection to the legacy config.

There is now also a change in behavior prior to the beta. Previously, the legacy configuration was applied directly upon repo creation, but now with the new system, it seems to happen asynchronously some time after repo creation (in my cases if was usually 1s afterwards).

This whole thing got me confused for some time and I am sure I do not understand everything but at least my tests were quite consistent so I am pretty sure the behavior is close to what I describe, but I would be happy to hop on a call with you to further discuss the topic.

[jamestran201]

Hey @netomi, thank you for your feedback and sorry for the inconvenience. If you're opted into the public beta, the steps below will unset any default configurations, and prevent them from being applied to new repos:

• Navigate to the security products settings page, the URL is something like https://github.com/organizations/<my-org-name>/settings/security_products

• This page lists all security configurations that have been created. The descriptions of each configuration can tell you whether that configuration will be used as the default for new repos

• Click into any configuration that is set as default for new repos

• Scroll down to the Policy section and unset the selected option for "Use as default for newly created repositories"

• Click on "Update the configuration"

I have another question based on what you described. We try to indicate which configurations are set as default for new repos on the security products settings page. What do you think about the indications given right now? Is it just fine, or easy to miss?

[netomi]

Hey James,

thank you for chiming in. Maybe there is some misunderstanding.

I have not opted in for the public beta, its just the default for all organizations that I am managing since 04/2024. Also no security configuration has been edited or created, everything is the same as it was setup automatically by GitHub itself.

In this case, the legacy configuration (the one available till 03/2024) is taken into account. The values of the config you still get via the rest API, but its not visible in the UI unless you opt out from the beta.

Now the change of behavior compared to 03/2024 is that the legacy configuration is not applied upon repo creation, but some time after as you can see from the logs screenshot above. Which is certainly not a problem when you do all configuration via the UI, but it breaks workflows if you use some tools that utilize the Rest API to create repos and change some settings after creation.

Regarding your question:

For the organizations that I manage, the security products settings page has obviously problems to show correct data as the settings are changed via the Rest API, not sure if your system can handle that in a meaningful way. But for somebody managing the repos via the web UI this looks like a useful tool afaict.

This is what I see in my orgs at times and I dont know what it actually means:

Edit:

I did not realize that you can change the Legacy configuration as well and make sure it is not applied to any new repository as well. There seems to be a UI glitch as the None option is not displayed, but when you click the first item, it selects None afterwards. That should solve my immediate problem afaict. It would need to be manually changed in all organizations though, so API support to disable that would be greatly appreciated as discussed in a call I just had with your colleagues.

[netomi]

just wanted to give some feedback on the newly added API support for managing code security configurations. That is exactly what we needed, tyvm!

[KateCatlin]

Awesome to hear, thank you @netomi!

[ChrisOwen273]

I have been playing around with this for the past few days and have two thoughts, not bugs I think but extra things that would be very useful for our use case.

Repository Admins can override

We have requirements to enforce policies on some of our repositories, the ability to repository administrators to opt out means we will have to invest time and effort into external solutions to keep re-applying the configuration, reporting on incompliance and internal processes to manage this.

Would be ideal if the managed settings showed up as "managed by your organisation administrator" and couldn't be overridden. Appreciate there are cases when you may want initial settings for best practice vs enforced settings for business requirements and this would need further thought to work out the logistics.

Configuration Auto Apply via Filtering

The filters to identify repositories to apply the configurations to is great, as is the setting in the configuration themselves to apply to newly created repositories.

For my use case it would ideal to be able define a filter within the configuration that will auto target and matching repositories.

For example, a filter set on a custom property of "repo_sensitivity", an existing repository has its sensitivity upgraded and now automatically meets the filter. As it is now, and as I understand the feature (so hopefully not getting this wrong) I would have to manually apply the configuration using the filter option, rather than it automatically happening as the criteria where met.

[KateCatlin]

Hey @ChrisOwen273! Thanks so much for reaching out. I have great news on both of these fronts.

Repository Admins can override
The ability to enforce configurations is coming soon - it's actually our next release in this space. I'll circle back here and post the changelog when it goes out.

Configuration Auto Apply via Filtering
This is something on our roadmap as well!

For now - In your specific use case, would you want to have that auto-application based on repo properties for only the one use case you've described, or are there multiple configuration/property pairings you'd want this to work for?

[ChrisOwen273]

That's great to hear @KateCatlin thanks for the quick confirmation on that!

I think that we would probably have multiple code configurations and each would have its own filter identifying a different "tier" or subset of our repos.

[KateCatlin]

Fascinating @ChrisOwen273, I'd love to dig in deeper about your use case. If you're willing to have a 25 minute chat, you can book me here!

If that's not in the cards right now, would love if you could share more about how many configurations you have today in your setup and how you target them differently in today's world. That would help inform us how our users are thinking about this!

[KateCatlin]

Hey @ChrisOwen273 just a heads up that configurations can now be enforced! Take it for a spin and let us know how it works for you! https://github.blog/changelog/2024-06-06-code-security-configurations-can-now-be-enforced/

[ChrisOwen273]

@KateCatlin I've had a look, seems to work well - thanks!

I would echo a couple of the points @ebickle raised as well:

• Applying to archived repositories would be great from a compliance perspective
• Enforcing CodeQL enforces the Default configuration, would be great if enforcing CodeQL allowed the repository admins to freely select from default or advanced configuration but not turn it off

[ebickle]

Hey everyone! I held off on submitting some feedback, but after trying the feature for awhile I have some thoughts and ideas!

• Unable to apply configurations to archived repositories: Applying a security configuration to an archived repository doesn't do anything - no visual feedback is displayed. Many security features are disabled for archived repositories (e.g. dependabot and code scanning) but others such as secret scanning still apply. In addition, if a repository is unarchived a security configuration would then be relevant. Could you please make it possible to apply security configurations to archived repositories? Any feature that's "unavailable" (e.g. dependabot) would remain inoperable (even if enforced) until the repository is unarchived.

• Configurations fail to apply when CodeQL advanced workflows exist: We initially had "Enforce" set to CodeQL and found that 10-20% of our repositories would have a status of "Failed" when we tried to apply the configuration. Once we set code scanning/CodeQL to "Not Set" the configuraitons were able to be applied - it looks as though configurations "override" any CodeQL advanced configuration. The way we'd like it to work is that CodeQL scanning should be able to be enforced - but enforced for either default setup or advanced setup. Deploying an advanced setup workflow should allow CodeQL default setup to be disabled (or automatically disabled).

• Default values: Currently there are only three states for each option - Enabled, Disabled, or Not Set. For enforced configurations where we want to give repository administrators control over a setting, we need to use "Not Set". The problem is this doesn't let us set a default value. For example, we'd like to have a default value of "Enabled" for Dependabot Updates (PRs) but allow repo administrators to disable it on adhoc basis if they'd like. Perhaps two new options - "Not Set - Enabled by Default" and "Not Set - Disabled by Default" or something similar would work?

• APIs: Is there an API for reading or changing the current configuration applied to a repository? The current UI with 25 repo pagination doesn't work well for us - we have over 7000 repositories so automating through API is necessary. I probably missed the REST API - let me know if it exists!

• Webhooks: Are there any webhook events that trigger when a configuration is applied or removed? I'd like to our our security automation GitHub app respond to change events and enforce setting specific security configurations where appropriate.

• Archived Status in Repository List UI: Could you please add the "Archived" text/label to the repository list on the security configurations page for archived repositories to match the UI for other areas of GitHub? E.g. "Private archived" in the "gray bubble" for a private-scope archived repository. Today it just shows "Private", which is inconsistent with the main GitHub repo list.

• Forking: Forks correctly have a configuration set (I assume the default for an org, but might also be the one from the fork source?) - this is great and something that was 'broken' previously. We had to write custom automation to apply security settings to forked repos because the defaults weren't set. Glad this is fixed!

• Move Additional Settings: Are there any plans to move additional settings into the security configurations feature? For example, there are many settings that can be globally defined on an org but still controlled on a repo level - "Autofix for CodeQL" being a good example. It would be nice to have these moved into the security configurations feature.

Thanks for all the hard work on this! It's been a lot of fun to play around with - glad we're able to stop users from disabling security now!

[KateCatlin]

Thanks @ebickle this is fantastic feedback! Some of this was on our radar and some of this is fresh so appreciate you writing it all up.

A few quick follow-ups:

• We'll release our API for configurations in ~2 weeks, so that's coming soon!
• We are indeed going to move all remaining settings from Global Settings to configurations, it's going to take a minute to migrate them all, but that's where we're headed.

In all, would love to find a time to talk with you in like 1 month after we've released the API and see how we can improve it for you. Want to find some time to chat around then? If so, you can choose whichever slot is most convenient for you here!

[KateCatlin]

Hey @ebickle quick follow up to let you know that our configurations API is now released! Let us know if you have any feedback!

[ebickle]

Thanks @KateCatlin! This looks like it will be incredibly useful.

I checked over the new APIs but didn't see anything that can be used to retrieve the security configuration for an individual repository. Is that something I missed? If it's not something that's available yet, having that API (Get code security configuration for a repository) would be very helpful - I'd like to write some automation that can swap security configurations for individual repositories in response to certain webhook events. We have a large # of repositories so listing all of them would be an 'expensive' set of API calls. 🙂

[KateCatlin]

Thanks @ebickle!

Unfortunately, we don't have the "retrieve the configuration per repo" functionality today; we only have "retrieve all repos per configuration." It was a conscious decision to skip it initially – we wanted to limit scope to release this functionality and start gathering user input. I'll share this feedback with the team though and we'll look into when we can ship that additional endpoint.

Can you share more about the webhook events that trigger configuration changes for you? I'd love to learn more about that. If you're open to a brief zoom call instead of a discussion thread, you can book me at whatever time is easiest for you here!

[molson504x]

@KateCatlin (tagging you since this is your thread, but feel free to tell me I'm wrong) - I love this feature and recommend it to clients a lot. HOWEVER I feel like the filtering is broken. I reproduced it in a test org I've got, where I applied a filter for archived=false and visibility != public, and well... the image speaks for itself:

I also tried this with the filter flipped (archived = false, visiblity = [private, internal] and got public repos back as well. This can be reproduced in multiple orgs, and testing additional configurations of the visibility filter produces bugged results as well (for example, visibility != [public, internal] shows this:

Thanks!

[KateCatlin]

Hey Matt! Thanks for reaching out! This is definitely a bug that we will look into asap. I'll post back here when we think we've got it fixed!

[archangelneo18]

Just out of curioisity, is there plans to make CODEQL as a required check or if CodeQL finds a vuln that it prevents the PR from being merged. bascially adding more grandualr rules at the global level for prevetion.

[KateCatlin]

Hey @archangelneo18! While we do have plans to move most of the settings currently in "Global Settings" to configurations, the features you mentioned exist under repo rulesets today. We have no plans at this time to move anything under rulesets to configurations!

Can I ask what your user story was here or how this came up? Now I'm curious!

[archangelneo18]

We deployed codeQL using the global settings. We are trying to make it a required check before a PR is merged, under the branch rule sets at the repo level it's trying to find a file that doesn't exist. At Global rule set it's looking for a repo with codeql. Ideally that's not scalable for us because we leverage multiple languages.

[SamMorrowDrums]

Hi @archangelneo18 I'm not quite sure what you tried here

under the branch rule sets at the repo level it's trying to find a file that doesn't exist.

But if you use the organization level rulesets that @KateCatlin linked to, you can now target by the default language of a repo (since last week).

So you could try creating separate rulesets for different target languages if that makes things easier for you? Even if you have to create a few, that should still work well.

[SamMorrowDrums]

I would also encourage you to contact support @archangelneo18 so we can help you better, as it sounds like you encountered an issue we might be able to assist with? https://docs.github.com/en/support/contacting-github-support