Updated at 9:13 PM ET, July 19, 2024
CrowdStrike is actively working with customers affected by a vulnerability discovered in a single content update for Windows hosts. Mac and Linux hosts were not affected. This was not a cyberattack.
The issue has been identified, isolated, and a fix has been posted. We are referring customers to our support portal for the latest updates and will continue to provide full, ongoing public updates on our blog.
We also recommend that organizations ensure they communicate with CrowdStrike representatives through official channels.
Our team is fully prepared to ensure the security and stability of CrowdStrike customers.
We understand the seriousness of the situation and are deeply sorry for the inconvenience and disruption. We are working with all affected customers to ensure systems are back up and running and able to provide the services their customers rely on.
We assure our customers that CrowdStrike is operating normally and that this issue is not affecting our Falcon platform systems. If your systems are operating normally, there will be no impact to their protection if the Falcon sensor is installed.
Below is the latest technical alert from CrowdStrike with more information on the issue and remediation steps organizations can take. We will continue to provide updates to our community and the industry as they become available.
summary
details
- Symptoms include hosts experiencing a bugcheck\blue screen error related to the Falcon sensor.
- Unaffected Windows hosts do not require any action as the problematic channel file has been restored.
- Windows hosts that are connected to the Internet after 0527 UTC will also not be affected.
- This issue does not affect Mac or Linux hosts.
- The channel file “C-00000291*.sys” with a timestamp of 0527 UTC or later is the returned (good) version.
- The channel file “C-00000291*.sys” with a time stamp of 0409 UTC is the version that has the problem.
- Note: It is normal to have multiple “C-00000291*.sys” files in the CrowdStrike directory – as long as One If a file in the folder has a timestamp of 0527 UTC or later, that is the active content.
Current procedure
- CrowdStrike Engineering was able to identify the posting of content related to this issue and reverse those changes.
- If hosts continue to crash and are unable to stay online to receive channel file changes, it is possible to use the workaround steps below.
- We assure our clients that CrowdStrike is operating normally and this issue is not affecting our Falcon platform systems.If your systems are operating normally, there will be no impact on their security if a Falcon sensor is installed. Falcon Complete and OverWatch services are not disrupted by this incident.
Query to identify affected hosts via advanced event search
Please see this Knowledge Base article: How to Identify Hosts That May Be Affected by a Windows Crash (pdf) or Log in to view the support portal.
Dashboard
Similar to the query above, a dashboard is now available showing affected channels, customer IDs, and sensors. Depending on your subscriptions, it is available in the console menu in either:
- Next Generation SIEM > Dashboard or;
- Investigation > Dashboards
- It is named as: hosts_possibly_impacted_by_windows_crashes
Note: The dashboard cannot be used with the Live button.
Auto-recovery articles:
Please see this article: Automatic recovery from blue screen in Windows cases in GCP (pdf) or Log in to view the support portal.
Workaround steps for single hosts:
- Reboot the host machine to give it a chance to download the return channel file. We highly recommend putting the host machine on a wired network (rather than WiFi) before rebooting as the host machine will be able to get a faster internet connection over Ethernet.
- If the host goes down again, then:
- Boot Windows into Safe Mode or Windows Recovery Environment
- Note: Putting the host on a wired network (as opposed to WiFi) and using Safe Mode with Networking may help resolve the issue.
- Go to the %WINDIR%\System32\drivers\CrowdStrike directory.
- Windows Recovery is set by default to X:\windows\system32.
- Go to the appropriate partition first (default is C:\), and navigate to the crowdstrike directory:
- A:
- cd windows\system32\drivers\crowdstrike
- Note: On WinRE/WinPE, navigate to the Windows\System32\drivers\CrowdStrike directory of the operating system folder.
- Locate the file corresponding to “C-00000291*.sys” and delete it.
- no Delete or change any other files or folders.
- cold boot host
- Shut down the host.
- Start the host from the stopped state.
Note: Hosts encrypted with BitLocker may require a recovery key.
Steps to work around public cloud or similar environment including virtual environment:
Option 1:
- Disconnect the operating system disk storage from the affected virtual server.
- Create a snapshot or backup of the disk volume before proceeding as a precaution against unintended changes.
- Attach/mount storage to a new virtual server
- Go to the %WINDIR%\System32\drivers\CrowdStrike directory.
- Locate the file corresponding to “C-00000291*.sys” and delete it.
- Detach the storage unit from the new virtual server
- Reconnect the persistent storage to the affected virtual server.
Option 2:
- Return to snapshot before 0409 UTC.
AWS Documentation:
Azure Environments:
User Access Recovery Key in Workspace ONE Portal
When this setting is enabled, users can retrieve their BitLocker recovery key from the Workspace ONE portal without having to contact the Help Center for assistance. To turn on the recovery key in the Workspace ONE portal, follow these steps. Please see this Omnisa article for more information.