CMM Deployment Methodology

Data Collection

The CMM uses a focus group methodology since it offers a richer set of data compared to other qualitative approaches. Like interviews, focus groups are an interactive methodology with the advantage that during the process of collecting data and information diverse viewpoints and conceptions can emerge. It is a fundamental part of the method that rather than posing questions to every interviewee, the researcher(s) facilitate(s) a discussion between the participants, encouraging them to adopt, defend or criticise different perspectives. It is this interaction and tension that offers advantage over other methodologies, making it possible for a level of consensus to be reached among participants and for a better understanding of cybersecurity practices and capacities to be obtained.

During the country review specific dimensions are discussed with the relevant group of stakeholders. Each stakeholder cluster is expected to respond to one or two dimensions of the CMM, depending on their expertise. For example, Academia, Civil Society and Internet Governance groups will all be invited to discuss both Dimension 2 and Dimension 3 of the CMM.

To determine the level of maturity, each Aspect has a set of indicators corresponding to all five stages of maturity. In order for the stakeholders to provide evidence on how many indicators have been implemented by a nation and to determine the maturity level of every aspect of the model, a consensus method is used to drive the discussions within sessions. During focus groups, researchers use semi-structured questions to guide discussions around indicators. During these discussions stakeholders need to be able to provide or indicate evidence regarding the implementation of indicators, so that subjective responses are minimised. If evidence cannot be provided for all of the indicators at one stage, then that nation has not yet reached that stage of maturity. With the prior consent of participants, all sessions are recorded.

 


Content Analysis

Content analysis (a systematic research methodology used to analyse qualitative data) is applied to the data generated by the focus groups. The purpose of content analysis is to design “replicable and valid inferences from texts to the context of their use”.

There are three approaches to content analysis. The first is the inductive approach which is based on “open coding”, meaning that the categories or themes are freely created by the researcher. In open coding, headings and notes are written in the transcripts while reading them, and different categories are created to include similar notes that capture the same aspect of the phenomenon under study. The process is repeated and the notes and headings are read again. The next step is to classify the categories into groups. The aim is to merge possible categories that share the same meaning. Dey explains that this process categorises data as “belonging together”.

The second approach is deductive content analysis which requires the prior existence of a theory to underpin the classification process. This approach is more structured than the inductive method and the initial coding is shaped by the key features and variables of the theoretical framework.4

 In the process of coding, excerpts are ascribed to categories and the findings are dictated by the theory or by prior research. However, there could be novel categories that may contradict or enrich a specific theory. Therefore, if deductive approaches are followed strictly these novel categories that offer a refined perspective may be neglected. This is the reason why the GCSCC research team opts for a third, blended approach in the analysis of the data collected by the Centre, which is a mixture of deductive and inductive approaches.

After conducting a country review, the data collected during consultations with stakeholders and the notes taken during the sessions are used to define the stages of maturity for each Factor of the CMM. The GCSCC adopts a blended approach to analyse focus group data and use the indicators of the CMM as criteria for a deductive analysis. Excerpts that do not fit into themes are further analysed to identify additional issues that participants might have raised or to tailor the Centre’s recommendations.

In several cases while drafting a CMM report, additional desk research is necessary to validate and verify the results. For example, stakeholders might not be always aware of recent developments in their country, such as whether the country has signed a convention on personal data protection. The sources that can provide further information can be the official government or ministry websites, annual reports of international organisations, university websites, etc.

For each Dimension, recommendations are provided for the next steps to be taken for the country to enhance its cybersecurity capacity. If a country’s capacity for a certain Aspect is, for example, at a formative stage of maturity then by looking at the CMM the indicators which will help the country move to the next stage can be easily identified. Recommendations might also arise from discussions with and between stakeholders.