Accelerating project planning with AI for stakeholder consultation
Project overview
Project
Empowering deeper collaboration in stakeholder consultation with AI
Stakeholder consultation is a critical path activity for major projects all over the world. Developing and maintaining communication with communities and organisations affected by new infrastructure is a fundamental aspect of developing a successful project, from gaining planning permission to ensuring that local needs are met. The process has a major impact on the project timeline and it is crucial that responses are securely collected, considered and responded to.
The stakeholder engagement tool captures all formal and informal consultation responses. It can be interrogated, shared and audited by project teams. (This dashboard contains sample data which is not from a live project.)
Over the past decade Mott MacDonald has revolutionised the approach to this through the creation of a stakeholder engagement tool hosted on its Consents Management platform. This is part of a comprehensive consent and approval management system spanning the entire project lifecycle powered by Moata digital solutions. The stakeholder engagement tool captures all formal and informal consultation responses. It can be interrogated, shared and audited by project teams, guarantees that all information is centralised and follows legal and data security requirements. What this could not do was analyse the content of these responses, compare them to technical documents and generate draft responses.
Until now.
Combining data science with stakeholder expertise
Development began in 2023 when Mott MacDonald‘s digital team saw the potential of harnessing large language models and generative AI to improve the quality and efficiency of work that involved huge volumes of data such as stakeholder engagement. Integral to this was feedback from Mott MacDonald’s Toronto office, which had been inspired by the way that AI was being used in political campaigns to analyse voter sentiment.
“Within the Consents Management platform we had very high-quality information that needed to be classified. It seemed like a great opportunity for harnessing AI,” says Matthew Fredericks, senior data scientist at Mott MacDonald.
For a large language model to be able to read through thousands of responses and classify them, it had to know what to look for. This is where the expertise of coders and data scientists such as Matthew had to be combined with stakeholder consultation specialists. “Until now we have used manual coding, which means that every response is read by a person, and the feedback is given a sub code based on the code framework,” explains Omar Bukleb, senior consultant for social impact at Mott MacDonald.
This code framework is both geographic and thematic. On a pipeline for example, the different sections are broken down into coded elements, so a 5km connection may be coded into ten 0.5m sections P1 to P10. Thematic analysis is then applied covering topics such as noise, visual amenity, traffic issues, wildlife and so on. Each of these themes would then be given an intensity score based on the depth of concern, and a positive or negative attribution. Teams of coders would spend weeks, if not months, analysing thousands of responses. “I have worked on projects where we had over 30,000 responses to just one section,” says Mike Logan, principal planner for transportation based in Canada. This means a lot of people are required to undertake the coding. Not only is this time consuming it is also subjective as each response is open to the interpretation of the reader. “With the invention of large language models we wondered if we could give our teams better tools for this analysis,” says Mike.
Identification of a positive attribution on the theme of health by the AI tool from an example consultation response
Instructing the AI
As part of its organisational enterprise system Mott MacDonald uses Microsoft Azure OpenAI, which provides the functionality of a large language model but in the company’s secure environment. The role of the data scientists was to provide a code framework that told the LLM how to identify and code the text from respondents. Key to this was creating a glossary of terms specific to projects and data being analysed. This was where the expertise of the stakeholder team was crucial. The better the glossary and instructions from the coders formed by the domain experts, the higher the rate of incidence between analysts and the AI.
Development was carried out using data from past projects, specifically hundreds of anonymised pieces of stakeholder feedback from work in water, highways and rail. Three experienced analysts worked on identifying and validating the data. At the same time Matthew and the team carried out blind tests comparing the AI classifications to that of the experts. Continuous improvements in the coding framework and how the data was organised led the correlation rate to increase to over 84%. Analysis of both human and AI responses showed that the AI classification was three times more consistent than that carried out by stakeholder analysts highlighting the subjectivity of language. The accelerated analysis with human review for quality assurance was many times faster than manual coding and review.
“One of the most important changes that we made during this process was partitioning the data that we gave to the model, basically not asking it to review too much at once,” says Matthew. The team also built in security measures to ensure that the AI was not hallucinating. “So for the intensity score for example we require that the AI returns a verified excerpt from the original text to justify its classification and explain why it has made the determination that it has,” says Matthew.
From a security perspective Mott MacDonald has a clear AI governance framework which included extensive risk analysis involving collaboration across legal, ethics, security, governance and technical teams.
Having AI functionality on top of the Consents Management platform takes it to a new level that makes it more efficient, more accurate and more consistent. Nothing is overlooked. Everything is addressed.Omar Deedat
Mott MacDonald technical director for stakeholder management
Delivering for clients
Confident in the security and capability of the AI functionality and maintaining human oversight with expert review of all outputs, the team has used it successfully on stakeholder consultations in the UK, Canada and North America. Cost and time savings are estimated to run into thousands of hours and therefore thousands of pounds. This is based on how it accelerates the process and reduces manual resources whilst upholding, and in some areas improving quality.
"Ultimately it is taking the place of highly intensive labour resources doing repetitive work,” says Richard Bowen, technical director for stakeholder management. “It removes bias. A human’s interpretation of what is in front of them is subjective and this is multiplied when thinking of the group. AI maintains objectivity, it is fast and it doesn’t need to take breaks.”
Thanks to the partnership between domain experts and data scientists AI has been harnessed to accelerate and improve the stakeholder consultation process, whilst maintaining human oversight and following a responsible governance process.
“In developing the tool over the past two years safety and governance guardrails have been key and working through the risk assessment was a very long process,” says Omar Deedat. “We are compliant with the EU AI Safety Act, everything is quality assured by our experts and clients are now benefitting from our investment in state-of-the-art technology.”
Looking ahead, Matthew and his colleagues continue to develop the system to enable it to do more, responsibly combining technical project expertise with data science and AI tools to benefit projects and communities.
Subscribe for exclusive updates
Receive our expert insights on issues that transform business, increase sustainability and improve lives.