On 26 July 2024, a security breach on a total of 13 accounts was detected. The Casper Association took immediate action to mitigate further risk.The unauthorized access occurred due to a vulnerability, which allowed an attacker to gain elevated access to resources via an elevation of privilege exploit. It is important to note that this was not related to the platform's architecture or consensus mechanisms.
The team became aware of the incident on 26 July 2024 at approximately 05:00 UTC. Subsequently, the root cause was identified on 26 July 2024 at approximately 13:00 UTC. Soon thereafter, a subset of validators agreed to pause consensus on 27 July 2024 - 07:50 UTC.The total amount of illicit transactions is estimated to be around USD6.7 million. A total of 13 wallets were affected. The Casper Association is working directly with the parties affected and it is expected that substantial amounts will be recovered, with a further reimbursement procedure being put in place to cover any potential shortfall.Due to pending investigation with the authorities we are unable to comment further at this time. Further information will be shared as it becomes available.In order to ensure this vulnerability is eliminated, a tool was developed to traverse the entire blockchain from genesis to tip and detect other instances. This thorough process took several days and yielded no other instances of exploitation.Multiple layers of assurance, including manual and automated testing, as well as third-party reviews of the network upgrade, provide confidence that the vulnerability has been eliminated. We are looking forward to working with our validator and broader community to resume growing and strengthening our network.We appreciate your trust and are working tirelessly to maintain it.The Casper Validators are meeting on Wednesday 31 July 2024 at 1400 UTC. In that meeting details of the upgrade will be provided. Once the validators accept the upgrade, they will then immediately resume consensus and minting of blocks.
After a thorough investigation, it was discovered that malicious actors exploited a vulnerability which allowed a contract installer to bypass the access rights check on urefs, thereby enabling them to grant the contract access to uref based resources. This escalation of privilege enabled illicit access including the ability to transfer tokens.
Timeline:
Resolution and Recovery:
When the severity of the attack was observed, swift action by a subset of Validators to pause consensus on the network prevented further exploitation of the vulnerability. This occurred on 27 July 2024 - 07:50 UTC.
The issue was quickly identified and a solution developed for Casper-node software to prevent any further exploit. In addition to patching the logic to prevent further exploit, new functionality was also added to the protocol upgrade logic to allow removal of illicit access from the attackers.
In order to ensure the vulnerability was not exploited prior to the incident, a tool was developed to traverse the entire blockchain from genesis to tip and detect other instances. This thorough process lasted several days and yielded no other instances of exploitation.
Multiple layers of assurance, including manual and automated testing, as well as third-party reviews of the network upgrade, provide confidence that the vulnerability has been eliminated. We are looking forward to working with our validator and broader community to resume growing and strengthening our network.
When the network was paused, there were two blocks within the current era. There is a higher technical challenge with resuming the network when paused in the middle of an era instead of at a switch block at the end of an era . The decision was made to remove the two existing blocks back to the previous era’s switch block.
The two removed blocks contained four (4) transactions which will effectively be orphaned, meaning their effects on the blockchain are removed. The affected deploys are as follows:
1 mint of a CEP-78 NFT, and 3 native transfers totalling 2,023,040 CSPR. Other than the four (4) transactions, which are known and can be replayed, no data was lost.
The upgrade will consist of a new Casper-node binary and normal configuration files. In addition, a global_state.toml file will be included which contains changes to global state which will be performed as part of the upgrade.
Because the network is paused, the staging of the upgrade and resumption of the network will require real-time synchronization of validators as opposed to the typical automated and synchronized upgrade. The validators will need to stage the upgrade and force run the version they staged.
Upon startup, the nodes will remove the two blocks, apply global state changes contained in the global_state.toml file, and once 66.7% or more of consensus stake reaches this point, the network will unpause and resume creating blocks.
This will result in the two blocks removed from the chain at height 3,329,417 and height 3,329,418 to be replaced with two new blocks with different hashes after the block chain resumes.
Block 3,329,417 with block hash: 2e24b81612c49379a5c3fb6717839f673f8d115f933271371f970973a8184849 contains 4 transactions that will no longer exist after the upgrade. Please make sure that your indexers correctly handle the removal of these transactions.
The deploy hashes for these 4 transactions are:
a72cf1519026ae088372f4a48eb41668238909237f7067f7644cddb9f7dcc8688c534259abc0aff3afc22eafb1c07c38756bbb3cbea4333ad736434ad0ea76aae719e6e3c41be8c22394bd7c91c87b15344c4e45253c5ca21fe65340ba57ac2725781e4c066bc68693888d7e1dbee639344e95dea6a422f9bc2695477b7745d7The commitment to the security and the integrity of the Casper protocol remains robust.
Your patience and understanding throughout this process is appreciated, and while the community would have preferred to not have to pause consensus, it was the right decision.
We will continue to provide updates and information as it becomes available and the investigation progresses. The community’s trust in the network is paramount, and we are dedicated to rebuilding and strengthening that trust through actions supporting the community in the coming days and weeks.
