The most important thing we can do before writing a single line of code is to clarify the division of responsibilities between us and the customer, as well as the classification of the solution and data. What requirements does the customer have, and what requirements comes from the government or other parties? Application security (AppSec) resources should already be part of the team in this phase to ensure that we meet security requirements and expectations.
We must also outline what we will do when a security incident occurs - what requirements are we facing, what is needed for successful disaster recovery, backup, and similar measures, and what will the consequences of downtime be for the customer?
1 - Roles and responsibilities
A lack of clarity in our responsibilities and those of others can have huge consequences for a project, so this must be clarified beforehand. It is especially important if companies other than us and the client are involved, as tasks and roles tend to fall through the cracks because everyone thinks “someone else” will handle it.
Bouvet conducts development projects in many different ways, where we take more or less responsibility for project management, planning, development, quality assurance, and not least the operation and management of the solution. We also involve our own, the client’s, and third-party equipment both during development and management of the solution.
Regardless of how the project is executed, it is important that we are aware of how responsibilities are divided. This should be regulated in the agreement with the client, so we must ensure that we:
Have control over our roles and responsibilities
Have contact points with all involved parties
Can follow up on deviations quickly so as to avoid misunderstandings or problems later in the project cycle
Note
The role delivery manager is responsible for security in the delivery, and is responsible for following up on any security concerns.
Operation and Management - Bouvet
If we are responsible for the operation and management of the solution in our infrastructure, our certification for ISO 27001 - Information Security will apply to it. This means that we have a greater overall responsibility for the security of the solution, and it is important that the delivery team is aware of this.
All resources managed by the delivery team must be handled in line with all other infrastructure in Bouvet, so the team must have routines for patching and maintenance or ensure that this is handled. Please note that any client resources and data requires their own backup regime - we don’t want to mix customers or our own internal data. Feel free to contact Internal IT & Security to see what they can assist with to simplify delivery and management.
Bouvet’s Statement of Applicability/Declaration of Application (SOA) addresses various controls related to information security and how we should relate to them. The SOA can be found in the internal management system. If we take on responsibility for operation and management, your regional quality manager can assist with advice and guidance to ensure that all responsibilities are covered.
Operation and Management - Client or Third Party
If we are only responsible for the development of the solution, it is important that we have defined the interface between us and the organization that takes over and continues to operate the solution:
How should handover occur
How do we ensure that the necessary hardware and systems have been set up and configured correctly
How do we ensure that all parties are aware of the requirements related to deployment, operational incidents, error corrections and similar
Tip
Document the roles and responsibilities and other relevant information in the source code system along with other documentation. This increases its visibility and and becomes the single source of up-to-date information for the whole team.
Most organizations operate with various classification levels for both data and systems. The classification level dictates how data is used, where it is stored, and who can access it. These are key requirements for any development project and must be clarified in advance.
Classification
Most organizations have routines and processes for data classification, typically distinguishing between these or similar levels:
Open
Internal
Confidential
Restricted
Depending on this classification, there may be different requirements for securing data. Open data typically has no restrictions, while data classified as “restricted” may have limitations such as:
strict requirements for access
information can only be processed or stored in approved systems
requires multi-factor authentication before access
has restrictions regarding the use of cloud services or the geographical location of where data is stored
This needs to be clarified with the client at the start of the project to ensure compliance. If the client does not use data classifications, one should still assess the sensitivity of the data to be processed to ensure that necessary measures are implemented.
Privacy
If the delivery team is to handle personally sensitive information, it is important that the team familiarizes itself with the requirements surrounding this. The Norwegian Data Protection Authority has published a guide for “Software development with built-in privacy” (in Norwegian) which provides useful insight into the issue.
Important
Norway has implemented GDPR into Norwegian law through Personopplysningsloven. All our deliveries has to consider the definitions and requirements which follows from this. Also remember that we are likely to be subject to data processing agreements that might carry even stricter requirements.
The purpose of the GDPR is to ensure the privacy rights of individuals through regulating the use of Personally Identifiable Information (PII). The GPDR has a number of requirements that define how PII can be used and for what. PII is defined as
“any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person”
Privacy may look like a complex and difficult topic, but generally
Any piece of information that can be tied to an individual is PII
Any use of PII requires a valid basis for processing it
The data subject must consent to the use of the data
Data minimization is a requirement - do not collect information unless it is needed
The information should have an end-of-life - a date where you consider deleting it. Do not store it for longer than needed.
The user has a right to be “forgotten” - also if you restore from a previous backup
If we process this type of information on behalf of clients, they will normally require that we sign a data processing agreement. If this is not in place, it must be discussed with the client’s representative.
Data for Use During Development and Testing
If data is used in connection with development and testing, it is important to consider classification and sensitivity. Development environments often have a different level of security than production environments, and this practically means that we cannot always use real data for development.
The team must check the need for anonymizing the data before it is used outside the production environment, so as to maintain the need for data amount, filling rate, and quality while ensuring that the client’s data does not go astray.
This is especially important if development occurs in Bouvet’s infrastructure but with a production environment located at the client’s site. In such cases, it is vital that Bouvet has documented routines regulating where and how data is stored and used, and how and when it should be deleted. This must be clarified in consultation with the client so that there is no doubt about responsibility, duties, and risk.
If a catastrophic event occurs, we must know who to contact and what requirements the solution and delivery team must adhere to. This not only includes typical availability requirements but also how long recovery can take, how it should be done, and what is an acceptable data loss.
Business Continuity Planning is not an IT-technical subject. But it is our responsibility as suppliers of an IT system to remind the customer that the system can become unavailable.
The answer from this planning will help describe the requirements for the solution’s robustness and security level, and is crucial for finding the right balance of cost and performance for the system.
Here it is important to consider
The criticality of the solution
Possible workarounds if the solution is unavailable
Consequences or extra work resulting from unavailability or when the solution becomes available again.
Customer Expectations
Do we have a Service Level Agreement with the customer that regulates uptime, availability, and similar, or does the customer have implicit expectations for this?
This must be clarified so that the team is aware of the consequences of downtime. In many projects, cloud components are used, where not all variables are under one’s control. Therefore, it is important early in the project to clarify the actual needs with the customer. We can ensure redundancy on all fronts if the customer wants it, but it will then cost accordingly.
Incident Management
All customers in Bouvet should have a defined contact point for incidents in Kunder (CRM). If there are other contact points from the team to the customer, such as a security operations center (SOC) or similar, it is also wise to document this so that incidents can be resolved as quickly as possible.
If an incident occurs and the customer or others need to contact the team, the usual practice is for the delivery manager to be the formal contact point in the Bouvet team.
Backup
Backup is important in all projects. Even if we do not have responsibility for the operation of infrastructure, source code systems, or other tools in many projects, we should be familiar common backup strategies, so that we can design and implement a solution that allows for these.
If we are responsible for operations, we are also responsible for ensuring that backups are performed. Common terms here are:
Recovery Time Objective (RTO) - acceptable time to achieve normal state after a failure
Recovery Point Objective (RPO) - acceptable data loss after a failure (measured in time)
The backup and recovery solution must be designed based on these requirements, and we must ensure that this is maintained. Together with the customer, we must determine:
How much?
Which data and systems should be subject to the backup regime. Can we differentiate between them?
How often?
Should we take backups once a week, every night, or every hour?
How far back?
How long should we store the backups?
A common approach to backup is to run:
Daily backups - stored for 30 days
Weekly backups - stored for 6 months
Monthly backups - stored for 2 years
It is also important to consider where the backups are stored, so that one is best prepared for catastrophic events. This can be solved by having offline and offsite backups, i.e., backups stored on, for example, tape and kept at a different physical location than other backups.
Test
Backups that are not tested have no value - implement a strategy that includes testing restore from backup!
Disaster Recovery
Disaster Recovery in the planning phase involves developing a plan for what should be done to return to normal state as quickly as possible. It can be useful to think of this as “actions on,” i.e.; “If X happens, we do Y”.
Recovery
It will not be necessary to recreate services in all disaster events. Often, less extensive and manual error correction can suffice. Regardless, one should always have a plan for complete recovery. Having this can save you in most situations.
Can infrastructure be recreated correctly and quickly enough?
Document resources, dependencies, and operational procedures
The use of Infrastructure as Code (IaC) will be a significant factor here
Data and databases
Total / bulk recovery: How do you recover large amounts of data / entire volumes?
Individual files: How do you recover a single file, table, row or column?
Support systems
Remember that support systems can play an important role in the overall system. These must also be replaceable or recoverable in the event of incidents
Examples: Git, CI pipeline, logging, and monitoring
Scenarios to Discuss
Deleted service: How do you restore a service that has been deleted?
Corrupt service: Do you repair or restore a VM or other services with problems?
Unavailable service: What happens if the services become unavailable? Here you need the definition of what is temporary / short-term downtime. Should you deploy a new service, do you already have one running as a hot-swap, or is it okay without one for a period?
Compromised security: How do you handle it when a password has been leaked in some way?
Expired password: How do you resume operations if a password or certificate has expired?
Compromised identity / service user: What do you do if a managed identity or a service user has been compromised?
Unavailable passwords: What do you do if the key vault service in the region you use goes down? Do you have backup and failover in another region?
Malware: How do you recover the system after a malware attack? Do you need a partial or total rebuild of all services?
Confidentiality breach: How do you handle someone gaining access to the service(s) you operate?
Compromised admin: Do you need to plan for what happens if the owner of the subscription deletes your entire Azure subscription?
Critical vulnerability: What happens when someone discovers a critical vulnerability in your application? It may be wise to have protocols ready for when you need to decide whether to shut down the service.
Misconfiguration is a common source of errors and vulnerabilities, and this also applies to tools. If possible, the team should standardize the use of tools and their extensions, ensuring that everyone follows a similar (and documented) workflow.
All development teams use various tools in the development process, and the selection will vary from team to team depending on personal preferences, technology choices, system and customer requirements, and much more.
a version control system for the code, typically git
a tool for CI/CD that can perform various tasks related to building, testing, or deployment
other services managed or consumed by the team, e.g., messaging services, file transfer services, or similar
These tools can significantly impact security and quality in deliveries, so it is important that the team considers how these are configured.
IDE
Most IDEs today allow the installation of extensions that provide support for new languages, formatting, linting, cloud services, and much more. These can significantly improve the productivity and efficiency of the team, but we must be cautious about what is installed.
As with everything else downloaded and run from the internet, we must consider the risk when using extensions. It is important that we are aware of what we download, where it is downloaded from, and who is behind it to avoid problems.
Version Control
Version control provides excellent control over all changes, but it is important that we use the tool properly.
Remember that source code is part of the project and must be considered in relation to disaster recovery and backups!
Security in the Source Code System
Many rely on solutions like Azure DevOps, GitHub, or similar that handle access control, reviews, and other functions related to confidentiality and integrity of the source code.
However, Git also has built-in functionality for signing commits, so that each commit can be traced to a person with a given key. This can be a useful tool for ensuring integrity and should be considered by the team.
Branching Strategy
A typical approach is to operate with a production branch, often main or master. This should be protected so that all changes occur in separate feature branches that are then merged via pull request with corresponding review from others in the team. The production branch then serves as the basis for all further deployments.
There are other more complex approaches as well, such as separate branches and version tagging. This is especially useful if maintaining multiple versions in different environments, needing the ability for hotfixes, or similar:
In this example, all developers work in their own feature branches against the develop branch, which is protected from direct changes. This is deployed to the dev environment to verify that everything works as it should.
When the team is satisfied with the state of develop, it is merged to test via a dedicated pipeline that handles version number tagging automatically. This pipeline may require approval to run, needing one person to start it and another to approve.
The test branch is deployed to the test environment, and when the customer is satisfied with what has been delivered, it is merged to the prod branch in the same way as to test. For both test and prod, we use the version number as part of the branch name, so we can have branches Test/v1 and Test/v2, corresponding with Prod/v1 and Prod/v2.
If hotfixing against prod is needed, this can be done against the relevant prod branch to quickly correct critical errors, and then the hotfix can be taken back to dev.
pre-commit
A tip is to use [pre-commit](https://pre-commit.com) to run all linting, formatting, and testing, and then use the same _pre-commit_ configuration in CI/CD. This will minimize maintenance, make it easy to test locally, and catch problems early.
Be aware that several of the points below require additional software. We currently do not have shared licenses for developers at Bouvet; this must be managed by each project depending on needs and requirements. If the team handles this on its own, be aware of license conditions and how tools work. Some tools, for example, send source code to their own servers for analysis, which is generally not allowed unless agreed upon with the customer.
Software Composition Analysis (SCA)
Software composition analysis (SCA) can be set up automatically as part of CI/CD. We have many dependencies on components made by others, so it is important to keep track of existing and newly discovered vulnerabilities in what we create.
Testing
Running tests in CI is beneficial for several reasons, but from a security perspective, there are specific tests that should be included.
Test all relevant endpoints for 401/403 responses
Test code that handles authorization (who gets to do what). It is advantageous if all authorization logic happens in a centralized place in the codebase.
Static application security testing (SAST) should be configured to run automatically as part of CI/CD. Consider whether a build should fail if the static code analysis detects serious weaknesses in the code or low test coverage.
Secret Scanning
Secret scanning - passwords, keys, and other sensitive information that should not be in the source code is an important tool that can be implemented in the version control system and in CI/CD. Some tools only provide alerts when secrets are found, while others can also invalidate the secrets in the services they are meant for.
A security checkpoint is a control point during a project where requirements must be met before proceeding.
Depending on the security level a delivery aims to achieve, it may be necessary to define mechanisms for assessing security at fixed points in the development cycle, known as security checkpoints.
These can be defined between logical phases in the project, for example, between the design and development phases, or when transitioning from development to the first release in production. Other and additional checkpoints can also be defined, entirely depending on the requirements the delivery team must adhere to.
In a study by IBM it was determined that defects in applications developed for the U.S. military cost significantly less to fix early in the process compared to later.
By implementing security checkpoints, it becomes easier to catch weaknesses early and to ensure compliance with security and quality requirements. A typical practice using checkpoints could include
creating a design of the project before the development cycle starts
the implementation should always follow the design
verification that the design and implementation actually match before going into production
