Real-world testbeds are developed to be a safe setting that ensures innovations work as anticipated, solve a problem, establish the use case, and provide the intended benefits and outcomes for the target audience.

The purpose of this tool always contains some form of testing innovation for verification and/or proof of concept.

Real-world testbeds often provide access to relevant stakeholders (such as citizens and potential customers) or the ‘target system’ the innovator seeks to enter (such as a public road or a hospital). Learning from the interaction of the tests and the target user or system is vital, as the tests can uncover mistakes, malfunctions, or raise questions that have not been previously considered. In turn, feedback from users can ensure more robust products and provide evidence of the functionality, safety, and usefulness of the innovation to support market deployment.

Verification and validation also provide the potential customer or user (for example a local authority or a public body) with essential information before making decisions regarding procurement, uptake, or investment. The evidence that comes from testing in the real world can contribute to reducing the risk of procurement of new technologies or other innovations.

• Geography: Amsterdam, Netherlands
• Facilitators: Massachusetts Institute of Technology (MIT) and Amsterdam Institute for Advanced Metropolitan Solutions (AMS)
• Public stakeholders: City of Amsterdam, Waternet – the public water utility of Amsterdam)
• Timeframe: 2017–2022

This real-world testbed is part of a five-year research programme on autonomous floating systems (‘Roboats’) in Amsterdam, designing the world’s first fleet (potentially) able to move people and goods, function as a portable, temporary infrastructure and gather data. Through testing the vessel in the canals of Amsterdam, the innovators were first able to verify that the technology – which had previously been tested in closed-off pools – functioned in the unpredictable environment of a lively city canal. They were also able to investigate how urban waterways in combination with the vessel can be used to improve city functions and quality of life.

Key success factors and lessons

• Proving safety and effectiveness supports testing and demonstrating in other cities.
• The real-world testbed showcased and improved control systems for autonomous navigation in urban environments and showed that new technology can revitalise 17th century infrastructure. The technology was first tested in Amsterdam before moving on to other similar canal and waterways cities such as Cambridge, UK, sharing the initial learning across the different urban environments.
• Using the real-world testbed to prove multiple uses and functions.
• The Roboat was used for a wide range of solutions to meet the city’s and technology developer’s respective needs. These include demonstrating technology such as 3D mapping of the urban infrastructure using artificial intelligence and ‘laserscapes’; functioning as temporary bridge infrastructure; water quality and pollution data gathering; and clearing the canals of waste. This partnership of testing new technologies – many of which had additional public service use cases – allowed for verification of multiple technologies in close collaboration with public bodies such as the water company and the city council.