On October 25, 2024, the Cybersecurity and Infrastructure Security Agency (CISA), alongside the US Federal Bureau of Investigation (FBI), and the Australian Signals Directorate's Australian Cyber Security Centre (ASD's ACSC) issued new guidance to help software manufacturers deploy software safely and securely. The guidance is intended for organizations deploying software across diverse customer systems, including mobile devices, laptops, and cloud services, and is designed to ensure that software rollouts are secure, controlled, and adaptable to different deployment environments.

According to the guidance, a secure deployment process should fulfill several objectives including ensuring reliability, reducing downtime, and minimizing security risks. The guidance explains that these objectives are achieved through effective quality processes and controlled deployment phases that include phased rollouts and feedback loops to continually refine the deployment process.

Phases in secure deployment

The guidance describes a phased deployment process that includes structured steps, from initial planning which involves establishing deployment goals, and requirements, and a clear operational risk assessment to map out potential threats, system dependencies, and security needs. During the development and testing phase, the guidance recommends rigorous testing methods, including unit, integration, and automated testing. According to the guidance, testing environments should closely simulate customer conditions to help catch issues early and organizations should conduct stress tests and intentionally attempt failure to identify weaknesses before deployment.

Deployment

In the deployment phase, the guidance advises controlled internal rollout and canary testing before full-scale deployment to enable teams to monitor system performance and address issues in a limited setting, minimizing the impact of unexpected failures. For urgent security patches, the guidance outlines the need for adaptable deployment speeds while allowing organizations to maintain 'emergency stop' mechanisms if critical issues arise during the rollout.

Post-deployment and further measures

The guidance also highlights the importance of continuous feedback and improvement post-deployment. It advises that insights from customers, system logs, and 'near misses' should feed back into the planning and development cycles. By learning from each release, the guidance explains that organizations can adapt to evolving security threats and technical demands.

To maintain resilience, the guidance recommends emergency protocols for effective incident response, including detailed response playbooks to ensure structured responses, including escalation paths, rollback procedures, and detailed steps for restoring system stability. For major updates, the guidance advises organizations to implement a structured customer notification plan, including pre-deployment notifications, real-time status updates, and clear communication during incidents. Finally, the guidance addresses customers who choose to stay on older software versions, often referred to as N-1 or N-2 releases. The guidance encourages organizations to enhance their deployment processes to ensure the latest versions are secure, stable, and attractive to encourage timely adoption among users.

You can read the guidance here.