The FDA is charged with the responsibility of overseeing the safety and effectiveness of medical devices. Commensurate with this responsibility is the challenge of how a regulator or third party (e.g. notified body) can assess the quality, integrity, and ultimately safety and effectiveness of increasingly complex and safety critical software systems used in these medical devices – which are under increasingly tight mandated time frames.

A challenge then, is to establish the means for a reviewer to relatively quickly establish confidence that a device is likely to perform as intended. We briefly explore technology behind model-based design to see how it might enable reviewer confidence in the behaviour and performance of medical device software.

What happens when it’s submitted?

To assess the software implementation of medical device features, a manufacturer submits software development life cycle documents and artefacts to FDA for review prior to being placed on the market. This is often voluminous, seldom uniform in organisation and content, reflects many person-years of design insight the reviewer does not have, and often provides little visibility into the actual device software. Confidence in device performance is largely derived from an audit of the development artefacts submitted. Since the product is theoretically ready for the market when this review occurs, any documentation discrepancies and errors discovered may affect confidence and subsequently trigger requests for additional information – which can be very costly for both the manufacturer and the regulator.

The standard IEC 62304, establishes requirements for a medical device software development life cycle. In its Annex C, it refers to IEC 61508 for tools, processes, and methods to consider when developing devices. Each of the IEC 62304 process steps, including associated verification activities (e.g. requirements traceability, consistency, etc.), is very human intensive in practice and fraught with opportunities for introducing errors – even with the aid of quality development process tools (e.g. configuration management, document management, etc.). Problems found in the integration process or later can be very expensive to correct. Evidence of software quality is typically based on testing, which too often provides little insight into design rigor, coding errors, or coverage. Modern techniques such as system- and component-level modelling and simulation of devices before they are implemented in software can reduce the costs associated with creating and verifying a device. While IEC 62304 addresses software development process activities, nothing is said about specific requirements analysis and functional design activities or measures of quality.

Model-based design/engineering

Modelling and simulation can provide unique and valuable insight into system behaviour. The term ‘model’ refers not only to the abstract representation of patient, physiology, or population behaviour (environment), but also the hardware and software to be implemented as the device. Today’s modelling capabilities provide the ability to simulate behaviour of the various components and the system as a whole before implementation is considered. Furthermore, models can be elaborated and viewed as a high-level software system development environment. Like the move from ASM to C, this allows more productivity and more emphasis on behaviour than implementation details. Code generation from models is akin to the code generation by compilers that has been in use for decades.

In addition to production code, such tools can generate evidence for the implementation steps to demonstrate compliance to a rigorous development process. By automating the artefact creation, one can focus more time on the requirement and behaviour analysis to improve the product quality. An effective process will seek to generate process compliance documentation naturally and automatically as a part of the development activities.

Metrics help justify development process and product quality, and the completeness and dependability of this evidence depends, in general, on two things:

  1. a measurably rigorous process and
  2. trustable tool chains that generate metrics while building and verifying and validating the product.

Here we consider the use of:

  1.  model-based design/engineering (MBD) for the process, in the context of IEC 62304
  2. the MathWorks based tool chain for building the product.

The FDA recognises IEC 62304 as an acceptable medical device software development life cycle process.

Metrics for IEC 62304

This table summarises the stages of model-based design mapped to the processes required by IEC 62304 and relevant artefacts, automatically captured during the verification process and published as reports.

The items highlighted in green can be automatically generated by Model-Based Design tools.

Conclusion

Here we present model-based design as a rigorous process with metrics that can serve to inform regulators or third parties on the quality, integrity, and ultimately safety and effectiveness, of software systems used in medical devices. Such metrics can form the basis for regulatory science, changing the discussion from an arbitrary quality process level to a concrete product quality level. Metrics could be presented in the form of assurance cases where the measures serve as evidence in arguments to justify product (safety, security, function, behaviour, etc.) claims. From an assurance case perspective some questions remain. For example:

  • are the measures we have discussed sufficient for medical device software?
  • for each quality metric, is a consensus on acceptability criteria among stakeholders possible?
  • do software quality metrics and corresponding acceptability criteria contribute to confidence? If so, can this confidence be measured in some consistent, objective, and/or quantitative manner?

We acknowledge limitations of accurately modelling reality (environment) as a way to test software systems in simulation. However, the process of modelling software behaviour is not only well understood, but is also robust and verifiable. Having said this, there is ample evidence that the MBD process is playing a key role in many mission/safety critical industry sectors – which includes medical devices.